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EXECUTIVE  SUMMA’*  V 


Preliminary  research  studies  were  conducted  to  investigate  the  effects  of  two  memory 
strategies  planning  and  flight  strip  management)  on  performance  in  the  Air  Traffic  Control 
(ATC)  environment.  Thre#*  experiments  were  conducted  using  novice  participants  (students 
currently  enrolled  in  an  avibdon  program  at  a  local  community  college)  to  determine  the 
most  valuable  methods  for  conducting  further  work  on  these  strategies  with  actual  air  traffic 
control  specialists  (ATCS’s).  Experiments  were  conducted  using  TRACON  11,  an  ATC 
simulator  for  the  personal  computer.  Participants  were  trained  to  become  fairly  proficient 
with  the  game  before  being  tested  in  the  experimental  conditions.  None  of  the  participants 
had  had  experience  with  TRACON  prior  to  these  sessions,  however,  performance  was 
observed  to  vary  greatly  between  individuals. 

Experiment  1  tested  the  effect  that  the  opportunity  to  plan  strategies  had  on  performance. 
Participants  were  tested  under  two  conditions,  one  which  encouraged  the  development  of 
planned  strategies,  the  other  which  discouraged  their  development.  The  results  did  not  reveal 
statistically  significant  differences  in  performance  between  these  two  conditions.  Participants 
did,  however,  indicate  that  their  perceived  level  of  thinking  and  concentration  was  higher  for 
the  condition  which  discouraged  planning. 

Participant’s  performance  varied  in  both  cond'  is.  More  than  half  of  the  participants  had 
completed  one  or  both  sessions  without  error.  Others,  however,  made  many  errors.  Post- 
hoc  analyses  were  conducted  to  determine  what  may  have  been  responsible  for  the 
differences  in  performance  between  these  two  groups  of  participants.  These  analyses 
revealed  that  the  group  of  participants  that  made  fewer  errors  indicated  a  lower  level  of 
stress  prior  to  entering  the  experiment  and  had  somewhat  more  experience  in  the  aviation 
program  than  did  those  who  made  more  errors.  Regardless  of  category,  error  scores 
increased  with  reported  stress  level  and  decreased  with  the  number  of  ground  courses 
completed.  Better  performance  was  also  correlated  with  lower  reported  levels  of  perceived 
workload,  lower  levels  of  "busyness,"  and  lower  levels  of  stress  following  the  test  sessions. 

It  was  also  proposed  that  individual  strategies,  which  were  not  directly  measured  in  these 
experiments,  may  have  affected  performance  variability. 

Experiment  2  tested  the  effect  that  an  increase  in  available  planning  time  prior  to  taking 
control  of  the  airspace  had  on  performance.  Participants  were  tested  under  two  conditions, 
one  of  which  allowed  for  more  planning  time  (5  minutes)  than  another  (2  minutes).  More 
complex  scenarios  were  chosen  for  this  experiment  since  the  majority  of  participants  had 
performed  so  well  previously. 

Overall,  performance  was  poorer  in  this  experiment  than  the  first.  Participant’s  reactions 
revealed  that  they  found  Experiment  2  to  be  very  difficult.  Reports  indicated  that  these 
sessions  were  very  stressful,  required  a  high  level  of  thinking  and  concentration,  kept 
participants  very  busy,  and  produced  a  high  level  of  perceived  workload.  All  of  the  post¬ 
session  reactions  differed  significantly  from  those  reported  in  Experiment  1.  The  results  did 
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not  reveal  a  statistically  significant  effect  of  planning  time  on  performance,  however. 

Having  more  time  to  plan  strategies  did  not  result  in  better  performance  than  having  less  time 
to  plan. 

As  in  Experiment  1,  performance  varied  between  individuals.  Some  did  very  well,  others 
very  poorly.  The  group  that  made  more  errors  reported  a  somewhat  higher  level  of 
perceived  workload  as  well  as  busyness  after  the  experimental  sessions  than  did  the  group 
that  made  fewer  errors.  Regardless  of  category,  performance  was  related  to  perceived 
workload.  As  performance  level  decreased,  perceived  workload  increased.  As  for 
Experiment  1,  it  was  possible  that  individual  strategies  were  responsible  for  differences  in 
observed  performance,  although  the  experimental  methods  used  did  not  allow  for  their  direct 
investigation. 

Experiment  3  tested  the  effect  of  flight  strip  manipulation  (note  writing)  on  memory  for 
critical  information  and  performance.  Correlations  revealed  a  relationship  between  flight 
strip  use  (the  extent  to  which  flight  strips  were  used  to  record  actions)  and  memory  for 
issued  commands  as  well  as  a  relationship  between  flight  strip  use  and  performance. 
Participants  who  wrote  more  on  flight  strips  during  the  session  tended  to  perform  better  and 
also  tended  to  remember  more  of  the  commands  they  had  issued  after  the  session  was 
completed.  Level  of  perceived  workload  was  also  negatively  correlated  with  flight  strip  use. 
Those  who  used  flight  strips  more  indicated  a  lower  level  of  perceived  workload. 

The  results  of  the  third  experiment  were  promising  in  that  they  suggested  an  important  role 
for  flight  strip  manipulation.  However,  future  research  will  be  necessary  to  determine 
whether  flight  strip  use  is  responsible  for  improved  recall  and  performance.  The  present 
results  cannot  rule  out  alternative  explanations.  For  example,  participants  who  performed 
better  may  have  been  those  who  used  better  strategies  which  allowed  them  to  be  more 
organized.  This,  in  turn,  may  have  freed  time  for  them  to  record  more  actions  accurately 
and  to  perceive  the  task  as  less  demanding. 

The  preliminary  studies  described  here  provide  information  as  to  the  way  in  which  the  effects 
of  planning  and  flight  strip  management  may  be  effectively  investigated  with  air  traffic 
controllers.  Individual  differences  in  performance  were  observed  throughout  training  and  the 
experimental  sessions,  despite  the  fact  that  participants  entered  the  sessions  with  the  same 
level  of  experience  using  the  TRACON  II  simulator.  One  possibility  is  that  the  individual 
strategies  used  by  participants  may  have  been  responsible  for  much  of  the  observed 
variability.  It,  therefore,  appears  worthwhile  to  directly  investigate  individual  strategies  in 
detail.  This  is  applicable  to  the  air  traffic  controller  population,  since  individual  differences 
in  controller’s  abilities  have  also  been  observed.  Once  an  understanding  of  the  strategies 
used  by  effective  controllers  is  achieved,  these  strategies  can  be  developed  in  others  through 
instruction  and  training. 
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1.  INTRODUCTION. 


1.1  PURPOSE. 

Research  was  conducted  to  investigate:  (1)  the  effects  of  planning  and  organizing 
information,  and  (2)  the  effects  of  physical  activity  (note  writing)  on  memory  in  the  Air 
Traffic  Control  (ATC)  environment.  Basic  psychological  research  has  found  that  these 
strategies  enhance  memory  and  both  have  been  cited  as  important  to  the  air  traffic  control 
specialist’s  (ATCS’s)  job.  However,  little  experimental  work  has  been  conducted  directly  on 
these  areas  involving  ATC  tasks. 

The  work  described  tested  novice  participants  (non-air  traffic  controllers  with  aviation 
experience)  on  their  performan  s  with  simulate  ATC  scenarios  using  TRACON  11  (Wesson 
International,  1990),  an  ATC  simulation  game  for  the  personal  computer.  Participants  were 
taught  the  basics  of  the  game  and  were  subsequently  tested  under  several  experimental 
conditions  in  order  to  determine  which  conditions  enhanced/hindered  performance.  The 
three  preliminary  experiments  described  in  this  paper  were  designed  to  serve  as  indicators  for 
the  areas  of  investigation  that  would  prove  most  useful  for  future  studies  with  actual  air 
traffic  controllers. 

1.2  BACKGROUND. 

Human  performance  errors  have  been  cited  as  the  primary  source  of  operational/system 
errors  in  the  ATC  environment  (FAA,  1987;  Kinney,  Spihn,  and  Amato,  1977).  These 
errors  have,  in  large  part,  been  attributed  to  memory  lapses,  yet  memory  issues  have  not 
been  widely  studied  in  this  domain.  This  work  concentrates  on  two  memory  issues  that  have 
emerged  as  highly  relevant  to  the  ATC  environment  (Garland  and  Stein,  1991;  Vingelis, 
Schaeffer,  Stringer,  Gromelski,  and  Ahmed,  1990;  Gromelski,  Davidson,  and  Stein,  1992; 
Stein,  1991).  These  involve  an  understanding  of:  (1)  the  mental  "picture"  or  conceptual 
organization  of  information  about  the  aircraft  and  airspace  under  control,  and  (2)  the 
usefulness  of  physical  activity  (i.e.,  manual  updating  of  flight  strips)  on  a  controller’s  ability 
to  retrieve  critical  information. 

Planning  and  organizing  have  been  described  as  critical  to  the  development  and  maintenance 
of  the  ATCS’s  "picture"  (e.g.,  Whitfield,  1979).  In  addition,  planning  and  organizing  were 
cited  by  ATCS’s  as  primary  characteristics  of  outstanding  controllers  (Gromelski,  Davidson, 
and  Stein,  1992). 

Reports  have  also  indicated  that  flight  strip  management  (e.g.,  marking,  arranging  their 
placement)  is  critical  to  the  ATCS’s  performance  (e.g.,  Hopkin,  1982).  Gromelski  et  al.’s 
report  recently  found  that  flight  strip  management  was  cited  by  controllers  as  the  most 
frequently  us^  memory  aiding  technique.  Flight  strip  management  and  planning  and 
organizing  strategies  are  likely  not  independent  of  one  another.  In  the  words  of  Hopkin 
(1990),  "Strips  help  the  controller  to  organize  work  and  resolve  problems,  to  plan  future 
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work,  and  to  adjust  current  work  in  accordance  with  future  plans"  (p.  63).  In  summary, 
these  memory  strategies  appear  promising  as  areas  for  further  study  in  the  ATC  environment. 

1.2.1  Memory. 

The  dominant  perspective  in  memory  research  suggests  that  memory  operates  as  an 
information  processing  system.  Three  stages  of  processing  are  involved.  First,  the  sensory 
storage  stage  maintains  an  exact  copy  of  the  information  for  a  very  brief  period  (up  to  1/2 
second)  in  the  receptors  of  the  eye  and  ear  (Lindsay  and  Norman,  1977;  Sperling,  1960, 
1963)  Ttien  in  short-term  or  working  memory,  information  is  maintained  for  several 
seconds  to  several  minutes.  Only  a  limited  amount  of  material  can  be  accura*'^ly  retained. 
Finally,  in  long-term  memory,  information  is  maintained  more  permanently.  Such 
information,  for  example,  includes  the  meanings  of  words,  memories  of  events,  memories  of 
established  plans  and  procedures,  and  visual  images.  Items  from  long-term  memory  can  be 
retrieved  or  "called  up"  to  working  memory  and  used  when  needed. 

A  central  concept  of  information  processing  theory  is  that  the  ability  to  store  and  retrieve 
information  from  memory  is  directly  related  to  the  strategies  or  learning  activities  that  one 
engages  in  during  information  acquisition.  As  Begg  and  Sikich  (1984)  described,  "You  get 
out  of  memory  what  you  put  into  it"  (p.  57).  Acquisition  strategies,  therefore,  affect  the 
ability  to  remember;  more  effective  strategies  promote  more  effective  retention  and  recall. 

1.2.2  Air  Traffic  Controllers  and  Short-term  Memory. 

Of  primary  importance  to  air  traffic  controllers  is  short-term  or  working  memory.  The 
ATCS’s  job  requires  that  varying  amounts  of  information  be  retained  in  memory  for  up  to 
several  minutes  while  being  continuously  updated.  For  example,  an  ATCS  must  remember 
each  aircraft’s  call  letters  and  continuously  remain  aware  of  changes  in  heading  and  altitude 
for  as  long  as  the  aircraft  is  under  his  or  her  control.  Since  forgetting  critical  information 
has  been  noted  as  a  leading  cause  of  system  errors,  controller  memory  enhancement  is 
essential  for  promoting  safe  traffic  management. 

The  literature  on  short-term  memory  emphasizes  that  it  is  limited  in  capacity.  Miller  (1956) 
proposed  that  only  five  to  nine  items  could  be  held  in  short-term  memory  and  recalled 
accurately.  This  effect  has  been  observed  consistently  for  static  memory  tasks,  that  is,  those 
in  which  a  specified  set  of  information  is  to  be  retained,  unchanged  in  memory  for  a  period 
of  time.  Air  traffic  controllers,  however,  work  with  information  that  is  dynamic  in  nature. 
Not  only  might  they  be  confronted  with  a  situation  in  which  more  that  five  to  nine  units  of 
information  must  be  recalled,  but  they  must  additionally  continue  to  update  memory  in  order 
to  acquire  and  maintain  pertinent  information  while  discarding  irrelevant  information. 
Pertinent  information  must  be  recalled  quickly  and  accurately  in  order  for  prompt  and 
effective  actions  to  be  taken. 
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Basic  research  has  identified  a  number  of  strategies  which  are  useful  for  enhancing  the  recall 
of  information  from  short-term  memory.  Two  strategies,  organization  of  to-be-remembered 
information  and  physical  activity  involving  to-be-remembered  information,  have  emerged  as 
relevant  to  the  ATC  environment  (Garland  and  Stein,  1991;  Gromelski  et  al.,  1992;  Stein, 
1992;  Vingelis  et  al.,  1990). 

1.2.3  Organization  of  Information. 

'Hie  first  strategy  concerns  the  role  that  the  planful  organization  of  incoming  information 
p.ays  in  enhancing  memory.  The  following  example  illustrates  this  strategy.  If  a  learner  is 
presented  with  a  string  of  12  letters  to  recall;  MTVFBIUSAIBM,  experiments  have 
demonstrated  that  he/she  will  remember  more  of  the  letters  if  the  string  is  organized  into 
meaningful  chunks,  MTV,  FBI,  USA,  IBM  (Crider  et  al.,  1989).  This  organizational 
framework  is  helpful  for  two  reasons.  It  increases  memorability  by  increasing  meaning  and 
it  reduces  the  number  of  individual  units  that  must  be  held  in  short-term  memory  from 
12  to  4. 

In  another  study.  Bower,  Clark,  Lesgold,  and  Winzenz  (1969)  also  demonstrated  that 
intentionally  planning  and  applying  organizational  strategies  during  information  acquisition 
significantly  increased  the  amount  of  information  remembered.  They  presented  two  groups 
of  subjects  with  a  list  of  1 12  words.  The  words  were  members  of  four  different  categorical 
hierarchies.  In  one  hierarchy,  for  example,  the  top  level  word  "MINERALS"  was  followed 
in  level  two  by  the  subcategories  "METALS"  and  "STONES."  In  level  three,  "METALS" 
was  further  divided  into  "RARE,"  "COMMON,"  and  "ALLOYS."  "STONES"  was  divided 
into  "PRECIOUS"  and  "MASONRY."  Three  to  four  examples  of  each  of  these  types  were 
then  included  in  level  four. 

One  subject  group  was  presented  the  entire  list  of  112  words  in  random  order  on  each  of 
four  acquisition  trials.  After  the  fourth  trial,  this  group  was  able  to  recall  an  average  of  65 
words.  The  second  subject  group  was  given  the  words  according  to  the  organized  hierarchy. 
On  the  first  trial  they  were  given  only  level-one  words.  On  the  second,  they  were  first  given 
words  from  level  one  followed  by  the  words  from  level  two.  On  the  third,  they  were  given 
the  words  from  level  one,  then  level  two,  and  then  level  three.  Finally,  on  the  fourth  trial 
they  were  given  all  the  words  beginning  with  those  in  level  one  through  to  those  in  level 
four.  This  group  remembered  an  average  of  1(X)  words,  significantly  more  than  the  group  to 
whom  the  words  were  presented  randomly  on  each  of  the  four  trials.  Such  results  suggest 
that  intentionally  learning  material  according  to  an  organized  and  meaningful  plan,  in  this 
case  "chunking,"  enables  more  information  to  be  recalled. 

The  examples  above  concern  verbal  material.  Memory  enhancement  has  also  been  obtained 
for  chunks  of  visual  or  imaged  information.  Visual  chunking,  like  verbal  chunking,  is  useful 
because  it  allows  a  number  of  items  to  be  associated  together,  thereby  reducing  the  number 
of  individual  units  that  need  to  be  remembered  (Begg,  1978).  Studies  by  Chase  and  Simon 
(1973)  have  demonstrated  the  usefulness  of  visual  chunking  to  recall.  TTiey  found  that 
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skilled  chess  players  were  able  to  more  quickly  and  accurately  recreate  previously  viewed 
chess  game  configurations  than  were  novices.  They  concluded  that  skilled  players  were 
better  able  to  organize  actual  chess  game  configurations  into  meaningful  chunks  which  then 
allowed  them  to  more  effectively  store  and  recall  the  relative  placement  of  individual  pieces. 
When  random  configurations  of  chess  pieces  were  to  be  recalled,  however,  skilled  players 
fared  no  better  at  reconstructing  the  board  than  novices,  presumably  because  the  information 
could  no  longer  be  grouped  in  a  meaningful  way. 

It  has  been  suggested  that  ATCS’s  use  similar  organizational  strategies  to  help  them 
effectively  remember  the  aircraft  under  their  control  and  their  relative  positions  to  one 
another  in  the  airspace  (Means,  Mumaw,  Roth,  Schlager,  McWilliams,  Gagne,  Rosenthal, 
and  Heon,  1988).  Means  et  al.  suggest  that  aircraft  are  grouped  according  to  salient 
characteristics  such  as  overflights,  geographic  proximity,  or  those  which  could  potentially  be 
involved  in  a  conflict. 

Chase  and  Simon’s  (1973)  work  indicates  that  organizational  strategies  improve  with  the  skill 
level  and  experience  of  chess  players.  This  has  also  been  suggested  for  ATCS’s  (Garland 
and  Stein,  1991).  Better  organizational  strategies  should,  therefore,  result  in  more  accurate 
recall  of  information  about  aircraft  and  should  also  allow  more  aircraft  to  be  handled 
effectively.  ATCS’s  who  use  organizational  strategies  should  show  a  higher  level  of 
competence  on  both  measures  in  comparison  to  those  who  do  not  use  strategies  or  to  those 
who  use  less  efficient  strategies.  The  "picture"  that  is  then  developed  by  controllers  through 
the  use  of  effective  chunking  should,  therefore,  allow  them  to  be  more  effective  in  dealing 
with  a  high  volume  of  traffic  and  should  also  allow  them  to  be  better  able  to  foresee  potential 
conflicts. 

Whitfield  (1979)  interviewed  controllers  to  get  their  descriptions  of  the  "picture." 

Descriptions  often  referred  to  the  "picture  as  a  plan"  (p.  22).  This  suggests  that  controllers 
develop  a  mental  representation  of  what  the  traffic  pattern  looks  like  and  how  it  would  look 
in  the  future  if  the  aircraft  proceeded  as  expected.  The  picture  is  disrupted  or  lost  if  reality 
does  not  match  these  expectations.  Whitfield  concluded  from  his  results  that  "An  unexpected 
situation  requiring  a  new  plan  had  caused  loss  of  the  picture"  (p.  22). 

1.2.4  Physical  Activity  and  Remembered  Information. 

A  second  memory  strategy  cited  as  useful  for  ATCS’s  involves  the  use  of  physical  activity 
with  the  to-be-remembered  information.  A  number  of  studies  in  basic  research  have 
indicated  that  recall  of  material  is  enhanced  through  physical  activity.  For  example, 
performing  action  phrases  (e.g.,  "tear  up  a  sheet  of  paper"  and  "blow  up  the  balloon")  was 
observed  to  enhance  recall  of  those  phrases  (Engelkamp,  1986;  Koriat,  Ben  Zur,  and 
Nussbaum,  1990;  Zimmer,  1986).  Memory  for  phrases  whose  actions  were  only  imagined 
was  not  as  high.  Engelkamp  (1986)  found  similar  results  for  memory  of  verb  pairs  (e.g., 
knock-push).  Free  recall  of  the  pairs  was  better  when  each  component  was  physically  acted 
out  rather  than  just  imagined  at  the  time  of  presentation. 
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Activity  has  been  cited  as  essential  in  order  for  the  ATCS  to  maintain  critical  information  in 
memory  (Hopkin,  1982).  Hopkin  (1991b)  has  argued  that  flight  strip  management  (writing 
on  them  or  physically  altering  their  placement  to  indicate  an  aircraft’s  current  status)  is  vital 
for  assisting  the  controller  in  maintaining  his  or  her  memory  of  performed  actions  and  to-be- 
performed  actions  as  well  as  for  helping  the  controller  to  keep  the  "picture. "  A  number  of 
reports  have  expressed  the  concern  that  controllers  will  be  more  likely  to  "lose  the  picture"  if 
this  type  of  activity  is  eliminated  (Hopkin,  1991a;  Jackson,  1989).  An  experimental 
assessment  of  the  usefulness  of  physical  activity  on  memory  in  the  ATC  environment  is 
essential  to  determine  how  much  of  a  performance  decrement  may  result  if  such  activity  is  no 
longer  involved. 

2.  GENERAL  METHOD. 

2.1  PARTICIPANTS. 

Thirty-two  novices  (non-air  traffic  controllers)  were  initially  recruited  to  serve  as  potential 
participants  in  three  experiments.  These  participants  were  students  from  the  Mercer  County 
Community  College  Aviation  Program  in  West  Windsor,  New  Jersey.  Given  their 
knowledge  and  interests,  it  was  expected  that  they  would  be  likely  to  have  the  kinds  of 
cognitive  skills  that  would  transfer  readily  to  the  ATC  environment.  The  experiments 
conducted  with  these  participants  were,  therefore,  expected  to  give  a  good  preliminary 
indication  of  the  type  of  subsequent  work  that  would  be  most  valuable  to  conduct  with  actual 
air  traffic  controllers. 

Data  from  at  least  16  participants  were  expected  to  be  necessary  for  each  experiment  in  order 
to  achieve  statistically  significant  results.  Sample  size  was  estimated  prior  to  the  training  and 
experimental  sessions.  The  number  of  participants  necessary  for  achieving  significant  results 
is  a  function  of  the  difference  between  the  test  conditions  and  the  variability  of  performance 
between  individual  participants.  A  greater  number  of  participants  is  required  to  detect  that  a 
difference  between  the  test  conditions  is  significant  if  the  difference  is  small  and  participant 
variability  is  high. 

Estimates  of  the  expected  difference  between  test  conditions  and  estimates  of  performance 
variability  are  usually  made  based  on  prior  research.  Little  prior  information  is  available, 
however,  on  novice  participant’s  performance  with  TRACON  II.  Estimates  about 
performance  were  instead  derived  from  the  observed  performance  levels  of  two  novice  users 
(one  of  whom  was  one  of  the  authors)  prior  to  the  development  of  the  experiments.  Their 
performance  was  monitored  over  sever^  sessions  and  was  assessed  by  the  number  and  kinds 
of  errors  made.  Errors,  for  purposes  of  these  experiments,  were  classified  by  severity  and 
assigned  point  scores,  as  described  in  detail  in  a  later  section.  Essentially,  the  more  severe 
the  error  the  higher  the  assigned  point  value.  Using  this  scoring  system,  an  estimate  of 
variability  (variance  =  6  error  points)  was  made. 
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The  smallest  difference  between  the  test  conditions  which  would  be  necessary  in  order  for 
significance  to  be  achieved  at  the  probability  level  of  .05  or  better  was  then  arrived  at  using 
the  variance  estimate  above  and  the  assumption  that  a  number  of  participants  from  the 
original  pool  of  32  would  drop  out.  It  was  determined  that  even  a  relatively  small  difference 
between  test  conditions  (3.2  error  points)  would  be  found  significant  if  16  participants  were 
tested.  This  number  appeared  suitable  since  it  meant  that  half  of  the  participants  could  drop 
out  without  jeopardizing  the  results. 

Students  were  expected  to  participate  in  a  training  phase  in  addition  to  one  or  all  of  the 
experiments.  Participants  were  tested  under  two  conditions  in  each  experiment.  Permission 
for  student  participation  in  this  research  was  obtained  from  the  Coordinator  of  the  Aviation 
Program  and  the  Assistant  Dean  of  Academic  Affairs  at  the  College  (see  appendix  A,  page 
A-1).  All  practice  and  experimental  sessions  were  conducted  in  one  of  the  Aviation 
Department’s  computer  rooms.  Each  session  was  run  during  each  allotted  1  1/2  hour  time 
block  twice  a  week.  This  time  was  designated  as  free  time  for  students.  Thirty  PC’s  were 
available  and  each  participant  worked  at  his/her  own  computer.  Participants’  data  were 
coded  to  maintain  confidentiality. 

Prior  to  beginning  the  experiments,  a  letter  outlining  the  purpose  of  the  project  was  sent  to 
the  Aviation  Coordinator  for  distribution  to  students  in  the  program  (see  appendix  A,  page 
A-2).  This  letter  acted  as  an  introduction  to  the  experiments  and  invited  interested  students 
to  attend  a  followup  session  in  which  further  details  were  discussed  and  questions  about 
participation  answered.  It  was  explained  that  this  study  would  be  used  as  the  basis  for 
further  research  into  the  cognitive  processes  associated  with  being  an  air  traffic  controller. 
Interested  students  were  then  provided  with  a  consent  form  describing  their  rights  as 
participants  in  the  study  (see  appendix  A,  page  A-3)  which  they  were  required  to  sign  before 
the  training  and  experimental  sessions  got  under  way. 

All  those  who  volunteered  to  participate  were  then  asked  to  fill  out  a  preliminary 
questionnaire  which  was  designed  to  elicit  relevant  information  about  factors  that  may  affect 
performance  in  the  current  study  (e.g.,  aviation,  computer,  and  video  game  experience,  or 
quality  of  vision).  The  preliminary  questionnaire  is  shown  in  appendix  A,  pages  A-4  and 
A-5. 


A  table  indicating  participants  by  code  number  and  their  participation  in  each  experiment  is 
shown  in  appendix  A,  page  A-6, 

2.2  EQUIPMENT. 

The  equipment  used  to  conduct  the  experiments  was  the  TRACON  II  ATC  Simulator  for  the 
IBM  PC  (Wesson  International,  1990). 

The  TRACON  II  Simulator  presents  aircraft  in  sectors  surrounding  Los  An^  »es,  San 
Francisco,  Chicago,  Miami,  and  Boston.  It  allows  for  variables  such  as  sector,  number  of 
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aircraft,  weather,  pilot  performance,  equipment,  and  number  of  potential  emergency 
situations  to  be  specified  directly.  TRACON  also  allows  scenarios  to  be  specifically 
programmed.  This  option  was  used  to  regulate  the  scenarios  and  to  ensure  that  all  student 
controllers  received  the  same  traffic  scenarios  which  included  arriving,  departing,  and 
overflying  aircraft.  Programmed  scenarios  were  used  in  all  three  of  the  experiments  as  well 
as  in  most  of  the  training  sessions.  In  each  of  the  programmed  scenarios  the  number  of 
relevant  airports  and  fixes  within  the  tested  sector  (Los  Angeles)  was  reduced.  The  rationale 
behind  this  was  that  fewer  names  and  locations  would  have  to  be  learned  and  would, 
therefore,  bring  things  to  a  more  manageable  level  for  novices. 

Three  out  of  five  airports  in  the  Los  Angeles  sector  (LAX,  Long  Beach,  Van  Nuys)  and  an 
average  of  7  of  25  fixes  were  involved  in  each  scenario.  All  variables  (e.g.,  weather,  pilot 
performance,  equipment)  were  set  to  "perfect."  Two  scenarios  were  programmed  for  each 
experiment.  The  two  scenarios  within  each  experiment  were  identical  to  one  another  except 
that  the  aircraft  call  letters  differed  between  them.  This  allowed  for  a  more  standardized  test 
environment  within  each  experiment  but  made  it  so  that  participants  would  be  less  able  to 
recognize  them  as  identical. 

Programmed  scenarios  were  created  by  running  random  TRACON  displays  and  storing  the 
generated  flight  strip  information  in  textfiles  so  that  they  could  be  edited  by  wordprocessor 
(WORDPERFECT).  Individual  aircraft  were  selected  to  include  a  range  of  aircraft  types. 
Each  aircraft’s  flight  information  was  edited  to  ensure  that  its  flightpath  included  only 
airports  and/or  fixes  from  the  selected  set.  The  times  at  which  aircraft  entered  the  sector 
were  distributed  so  that  a  range  of  about  three  to  about  eight  aircraft  would  be  present  in  the 
airspace  at  different  points  in  the  scenario.  (The  number  of  aircraft  present  in  the  scenario 
cannot  be  controlled  precisely  since  the  ability  of  participants  to  successfully  manage  the  flow 
of  traffic  into  and  out  of  the  airspace  also  affects  this  variable).  Once  editing  was  completed, 
the  simulation  files  were  converted  back  to  the  DOS  editing  system  so  that  TRACON  could 
read  and  execute  them. 

2.3  TRAINING. 

It  was  expected  that  participants  would  require  a  reasonable  amount  of  training  with  the 
simulator  to  learn  how  information  is  presented  and  how  commands  are  issued.  Training 
was  expected  to  be  distributed  in  four  sessions,  each  conducted  on  a  separate  day  in  order  to 
reduce  fatigue  as  a  factor  in  learning.  Distributed  practice  has  long  been  considered  more 
effective  than  massed  practice  for  information  retention  (e.g.,  Madigan,  1969;  Melton, 

1967).  Allotted  time  per  session  was  expected  to  be  1  1/2  hours.  After  the  training  sessions 
began,  however,  time  constraints  were  realized  and  session  length  was  reduced  to  a 
maximum  of  1  1/4  hours.  This  time  reduction,  coupled  with  the  observed  performance  of 
participants  in  ongoing  training  sessions,  required  that  the  number  of  training  sessions  be 
extended  from  four  to  six. 
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During  the  first  training  session,  participants  viewed  three  20  minute  demonstration  scenarios 
that  illustrated  the  basics  of  TRACON.  They  were  able  to  observe  the  way  in  which  10 
aircraft  were  handled  within  the  Los  Angeles  sector.  Key  functions  used  to  issue  commands 
were  described,  problem  situations  (crashes,  handoff  errors,  separation  errors)  were 
illustrated,  and  feedback  regarding  errors  was  demonstrated. 

Along  with  the  information  provided  by  TRACON,  the  researchers  were  available  to 
elaborate  on  these  topics  and  to  answer  questions  during  the  training  sessions.  A 
training/reference  manual  was  developed  which  included  all  of  the  information  necessary  to 
control  aircraft  for  the  current  experiments.  The  manual  included  maps  of  the  airspace, 
information  about  airports,  and  information  about  key  commands.  It  is  presented  in  appendix 
A,  pages  A-7  through  A-17.  This  10  page  document  was  designed  to  use  visual/graphic 
depictions  in  lieu  of  written  dialogue  as  much  as  possible.  The  use  of  visual  presentations 
was  based  on  interviews  conducted  in  1991  with  FAA  developmental  controllers  (Gromelski 
et  al.,  1992)  who  recommended  the  use  of  visual  presentations  as  much  as  possible  for 
training  aids.  In  addition  to  this  extensive  manual,  a  "quick  reference"  card,  summarizing  all 
critical  commands  and  key  functions,  was  provided.  This  is  presented  in  appendix  A,  page 
A- 18.  An  overview  of  the  training  manual  was  provided  during  the  first  session. 
Additionally,  color  coded  patches  were  placed  over  the  six  keys  used  to  issue  control 
commands  so  that  they  could  be  targeted  quickly.  Participants  were  then  given  a  few 
minutes  of  hands-on  experience  with  a  randomly-generated  TRACON  scenario. 

During  the  second  training  session,  students  practiced  with  additional  randomly-generated 
TRACON  scenarios.  After  observing  participants  work  on  these  scenarios,  it  became 
apparent  that  a  more  focused  approach  to  training  would  be  appropriate.  New  training 
scenarios  were  specifically  programmed  to  instruct  students  how  to  handle  each  of  the  three 
types  of  flights  (arrivals,  departures,  and  overflights)  that  could  be  encountered.  This  was 
intended  to  provide  a  more  systematic  approach  to  training. 

During  the  third  training  session,  students  worked  with  each  of  three,  low- volume 
(9  aircraft/20  minutes)  programmed  scenarios;  one  for  arrivals,  one  for  dqiartures,  and  one 
for  overflights.  In  the  fourth  session,  participants  also  had  the  option  to  work  with  higher 
volume  arrival,  departure,  and  overflight  scenarios  (11  to  13  aircraft/20  minutes). 

Participants  either  chose  to  work  with  the  low  volume  or  high  volume  scenarios, 
depending  on  their  perceived  level  of  competence  and  the  level  observed  by  the 
experimenters. 

During  the  fifth  training  session,  students  worked  with  scenarios  that  contained  a  mix  of  the 
three  different  types  of  flights  (12  aircraft/30  minutes),  thereby  making  the  scenarios  more 
realistic  and  more  similar  to  those  they  would  observe  during  experimental  testing.  The 
experimenters  also  provided  individual  instruction  to  those  participants  who  demonstrated 
difficulty  in  managing  the  traffic. 


8 


During  the  sixth  and  final  training  session,  students  worked  with  more  of  the  mixed-flight 
scenarios.  Participants  were  also  instructed  as  to  how  their  performance  scores  would  be 
recorded  during  the  upcoming  experiments.  They  were  also  given  time  to  practice  the  self¬ 
scoring  procedure.  Scoring  is  described  below  for  each  experiment. 

Sixteen  participants  from  the  original  pool  of  32  remained  after  the  final  training  session. 
None  of  them  had  had  any  experience  with  TRACON  prior  to  their  participation  in  this 
study.  Thirteen  participants  then  went  on  to  complete  the  first  experiment.  Student  attrition 
was  due  to  time  constraints  (e.g.,  conflict  with  the  start  of  other  classes  or  with  flight  time, 
or  travel  to  and  from  a  job)  or  loss  of  interest  over  the  six  training  sessions.  The  number  of 
participants  completing  the  first  experiment  was,  therefore,  lower  than  expected  would  be 
needed  to  achieve  statistically  significant  results. 

An  observational  assessment  of  performance  during  training  indicated  that  participants  varied 
widely  in  their  ability  to  manage  the  traffic  in  the  programmed  scenarios.  Some  frequently 
experienced  separation  conflicts,  handoff  errors,  and  even  crashes,  while  others  were  able  to 
perform  without  making  any  errors  at  all  regardless  of  the  complexity  of  the  scenarios.  One 
of  the  experimenters,  a  retired  ATCS,  observed  and  talked  with  participants  during  the 
sessions.  He  commented  that  those  who  appeared  more  advanced  tended  to  look  for  control 
techniques  to  manage  traffic  more  effectively,  beyond  the  basic  control  techniques  that  had 
been  specifically  instructed.  For  example,  some  of  the  more  advanced  individuals  were 
transferring  information  onto  the  scope  via  data  tags.  This  allowed  them  to  type  in  the 
destination  of  each  aircraft  below  its  call  letters  so  that  they  would  not  need  to  refer  to  the 
flight  plan  lists.  In  addition,  the  use  of  the  "flightplan"  command  to  monitor  an  aircraft’s 
route,  and  clearing  aircraft  direct  to  the  airport  instead  of  issuing  headings  to  each  aircraft, 
appeared  to  leave  these  participants  with  more  time  to  plan  their  actions.  These  participants 
found  this  information  on  their  own  using  the  "HELP"  feature.  All  in  all,  the  more 
advanced  participants  were  more  proficient  and  developed  more  techniques  to  help  them 
make  fewer  errors.  These  participants  appeared  highly  motivated  and  were  concerned  with 
their  performance.  They  knew  that  separation  conflicts  took  away  from  their  total  score  and 
they  strived  hard  not  to  make  errors.  The  slower  students  did  not  appear  able  to  grasp  these 
various  techniques  until  they  were  taught  more  specifically  about  them  during  subsequent 
sessions. 

3.  EXPERIMENT  1:  EFFECT  OF  PLANNING  OPPORTUNITY  ON  SIMULATED  ATC 
PERFORMANCE. 

3.1  PURPOSE. 

The  purpose  of  the  first  experiment  was  to  investigate  how  the  opportunity  to  plan  and 
organize  information  about  aircraft  (e.g.,  heading,  altitude,  destination)  affects  the  ability  to 
control  those  aircraft.  Performance  was  compared  between  a  condition  which  encouraged  the 
use  of  planning  and  organizational  strategies  and  one  which  discouraged  their  use. 


9 


In  the  ATC  environment  planning  can  be  encouraged  by  allowing  information  about  future 
traffic  situations  to  be  available  and  studied.  Early  access  to  information  would  allow 
participants  the  opportunity  to  make  predictions  about  where  aircraft  will  enter  the  airspace, 
whether  potential  conflicts  might  occur  with  other  aircraft,  and  whether  the  aircraft  will 
require  substantial  future  control  commands  (arrivals)  or  not  (overflights).  Appropriate 
actions  can  be  selected  in  advance  of  the  event’s  occurrence. 

Alternatively,  planning  can  be  discouraged  by  keeping  participants  from  having  early  access 
to  information  about  aircraft  and  flight  plans.  This  situation  reduces  the  time  available  for 
organizing  information  and  for  making  predictions  about  potential  conflicts  and  decisions 
about  how  to  handle  traffic  problems  by  restricting  participants  to  working  with  each  aircraft 
as  it  appears  on  the  scope. 

3.2  METHOD. 

3.2.1  Participants. 

Thirteen  students  attended  both  sessions  of  Experiment  1.  Responses  to  items  on  the 
preliminary  questionnaire  indicated  that  the  ages  of  the  participants  ranged  from  18  to  26 
(mean=20.54,  SD=2.54).  They  had  completed  from  one  to  six  (mean=2.62,  SD=1.42) 
semesters  in  the  aviation  program  and  from  one  to  eight  (mean=3.08,  SD=2.47)  ground 
courses  prior  to  the  start  of  the  experiments.  Flight  experience  varied  greatly,  ranging  from 
0  for  one  participant  to  250  hours  for  another  (mean=101.69,  SD=86.4).  Two  of  the  13 
participants  also  indicated  that  they  had  some  aviation  experience  prior  to  attending  Mercer 
County  College  but  did  not  elaborate  on  what  that  involved. 

Participants  also  rated  themselves  (1  slowest,  10= highest)  in  a  number  of  different 
categories  pertaining  to  more  general  personal  factors  that  may  contribute  to  performance  on 
the  tasks.  Participants  rated  their  level  of  computer  experience  (mean  =5.00,  SD=1.96), 
level  of  video  game  experience  (mean=7.85,  SD=1.54),  and  quality  of  vision  (mean=8.85, 
SD=1.83).  None  of  the  participants  indicated  any  color  vision  deficiency.  They  indicated 
high  agreement  in  their  willingness  to  volunteer  for  the  study  (mean=9.85,  SD=.54)  and  in 
the  assessment  of  their  health  (mean =9.54,  SD=.86).  None  of  the  participants  indicated 
that  they  were  currently  taking  any  medications  that  would  have  interfered  with  their  mental 
or  motor  abilities.  They  did  indicate  variable  levels  of  recent  stress  (mean =4.23, 

SD=2.47).  Participants  also  assessed  how  motivated  they  were  to  be  involved  in  the 
experiments.  This  question  was  worded  so  that  a  response  of  1  indicated  the  highest  level  of 
motivation  and  10  the  lowest.  Participants’  responses  were  generally  high  (mean =1.69, 

SD  =  1.52). 
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3.2.2  Test  Conditions. 


3.2.2. 1  PLANNING  condition. 

Under  the  PLANNING  condition,  participants  were  tested  during  a  30-niinute  session  over 
which  12  aircraft  were  presented.  Prior  to  the  start  of  the  experimental  session,  subjects 
were  given  a  brief  10-minute  practice  session  so  that  they  could  become  familiar  with  how 
this  advance  information  could  be  used  to  help  them  plan  actions  for  the  upcoming  scenario. 
Paper  flight  strips  and  the  paper  map  of  the  airspace  which  included  the  names  of  airports 
and  fixes  were  made  available.  Flight  strip  information  included  the  aircraft’s  call  letters, 
altitude  and  speed,  location  at  which  it  would  enter  the  airspace,  and  destination.  Also 
included  was  a  three-letter  abbreviation  indicating  whether  the  aircraft  was  an  arrival  (TWR), 
departure  (T/0),  or  overflight  (CTR)  as  well  as  the  time  at  which  each  aircraft  would  enter 
the  airspace.  Participants,  therefore,  had  information  as  to  when  they  would  need  to  make 
contact  with  each  aircraft,  its  entering  location  and  destination,  as  well  as  its  current  altitude 
and  speed  before  the  session  began.  Flight  strips  for  the  five  aircraft  used  in  the  practice 
session  are  shown  in  appendix  B,  pages  B-1  and  B-2. 

Participants  were  verbally  instructed  as  to  how  they  could  plan  ways  to  handle  aircraft  during 
the  session.  Appendix  B,  page  B-3  describes  the  type  of  instruction  that  participants 
received.  It  was  suggested  that  they  plot  the  flightpaths  of  aircraft,  anticipate  potential 
conflicts,  and  think  about  how  they  would  direct  those  aircraft  to  avoid  errors  while 
maintaining  efficiency  (i.e.,  not  deviating  too  far  from  each  aircraft’s  most  direct  route). 
Participants  were  given  an  opportunity  to  write  notes  to  help  remind  them  how  they  would 
direct  traffic  during  the  session.  During  this  practice  session,  participants  were  given 
guidelines  for  thoughts  and  activities  expected  to  be  helpful  in  directing  traffic  in  an  efficient 
and  organized  way.  These  were  provided  by  an  experienced  ATCS  who  was  also  one  of  the 
experimenters.  Specific  planning  and  organizing  activities  for  the  scenarios  were  not  taught. 
It  was  left  up  to  individual  participants  as  to  how  they  would  develop  the  specific  strategies 
that  they  felt  would  best  help  them  direct  the  traffic. 

After  working  through  the  practice  session,  participants  were  given  flight  strips  for  the  12 
aircraft  they  were  to  direct  during  the  experimental  session  (see  appendix  B,  pages  B-4  and 
B-5).  They  were  then  given  10  minutes  to  plan  strategies  for  managing  the  traffic.  They 
were  instructed  in  this,  as  in  all  experiment^  conditions,  to  minimize  errors. 

Under  this  condition,  the  pending  flight  strips  provided  by  TRACON  on  the  computer 
screen  were  continuously  visible.  This  provid^  another  source  of  information  as  to  which 
aircraft  would  be  due  in  the  sector  within  the  next  few  minutes.  This  information  could  also 
be  used  as  a  reference  for  planning.  Under  this  condition,  the  boundary  of  the  airspace  was 
clearly  outlined  by  dashed  lines  so  that  participants  would  be  readily  able  to  determine  when 
an  aircraft  would  be  entering  the  sector.  Participants  also  had  access  to  the  "fligh^lan" 
command,  which  showed  where  each  aircraft  would  head  if  no  intervening  commands  were 
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issued.  This  provided  a  reminder  about  an  aircraft’s  direction  and  destination  during  the 
session  itself. 


3.2.2.2  NO-PLA] 


CICIIC 


[G  Condition. 


Under  the  NOPLANNING  condition,  participants  did  not  have  access  to  pap»  flight  strips 
either  before  or  during  the  session.  Pending  flight  strips  provided  by  TRACON  woe  also 
removed  from  view  during  the  test  session.  Only  the  active  flight  strips  were  visible  on  the 
screen  during  the  session.  Participants  were  also  unable  to  use  the  "flightplan"  command 
and  did  not  have  the  boundary  of  the  airspace  indicated.  These  changes  were  made  by 
editing  TRACON’s  initialization  file  which  allows  for  color  changes  such  as  "blacking  out" 
and  for  turning  other  options  (boundary,  flightplan)  to  the  "ofr  mode.  A  10  minute  practice 
session  was  provided  so  that  participants  could  become  familiar  with  working  under  this 
condition. 


3.3  DESIGN. 

Participants  were  tested  according  to  a  2  x  2  mixed  design  with  two  levels  of  each  factor  (see 
table  3-1).  The  PLANNING  and  NO-PLANNING  conditions  served  as  the  two  levels  of  the 
within-subjects  factor  (PLAN).  Students,  therefore,  participated  under  both  of  these 
conditions.  The  order  in  which  the  participants  worked  under  each  condition  was  the 
between  subjects  factor  (ORDER).  One-half  of  the  participants  worked  under  the 
PLANNING  condition  on  the  first  day  of  testing  and  one-half  worked  under  the  NO¬ 
PLANNING  condition.  On  the  second  day,  participants  worked  under  the  alternate 
condition.  Main  effects  of  PLAN  and  ORDER  as  well  as  the  effect  of  die  interaction  wm 
analyzed  by  an  analysis  of  variance  (ANOVA)  using  SPSS.  An  ANOVA  is  a  statistical 
procedure  used  to  determine  whether  the  differences  between  two  or  more  means  are 
significant. 


TABLE  3-1.  DESIGN  FOR  EXPERIMENT  1  (Fhrticipants  worked  under  both 
the  PLANNING  and  NO-PLANNING  conditions  in  the  order 
indicated) 


PLANNING 

NO 

PLANNING 

ORDER  -  1 

1st 

2nd 

ORDER  »  2 

2nd 

Ist 

Two  programmed  scenarios  were  created  as  described  in  the  Equipment  section  above. 

These  scenarios  were  the  same  excqit  for  the  call  letters  of  the  aircraft.  Scoiarios  contained 
six  arrivals,  four  dqiartures,  and  two  overflights.  The  order  in  which  the  scraarios  wm 
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worked  with  by  participants  was  counterbalanced  across  individuals  to  reduce  the  possible 
confounding  effect  of  different  scenarios  on  the  conditions  under  investigation. 

3.4  PROCEPVRE. 

Experiment  1  was  conducted  on  2  days.  On  the  first  day,  the  participants  were  randomly 
assigned  to  either  the  PLANNING  or  NO-PLANNING  condition.  Six  were  instructed  by 
one  of  the  researchers  as  to  how  to  proceed  under  the  PLANNING  condition  and  worked 
through  the  corresponding  10-minute  practice  scenario.  In  another  room,  the  other  seven 
participants  were  instructed  by  the  second  researcher  on  how  to  proceed  under  the  NO¬ 
PLANNING  condition.  The  whole  group  then  reconvened.  The  NO-PLANNING  group 
worked  with  their  10-minute  practice  scenario  while  the  PLANNING  group  spent  this  time 
previewing  the  12  flight  strips  and  planning  strategies  for  the  upcoming  test  session. 

The  same  procedures  were  followed  on  the  second  day  of  testing  in  which  participants 
worked  under  the  alternate  condition. 

3.5  PERFORMANCE  MEASURES. 

The  dependent  variable  was  the  measure  of  performance  obtained  by  an  error-point  total. 

This  total  was  based  on  the  error  information  provided  by  TRACON.  Errors  included 
crashes,  separation  conflicts,  missed  approaches,  and  handoff  errors.  Errors  were 
categorized  by  severity  and  given  different  point  values  according  to  their  severity.  Since 
crashes  are  the  most  severe,  each  was  assigned  10  points.  Separation  conflict  errors  and 
losing  aircraft  off  the  scope  were  each  assigned  5  points,  missed  approaches  and  handoffs 
made  at  the  wrong  altitude  were  each  assigned  3  points,  and  other  handoff  errors  were 
assigned  2  points.  Feedback  on  these  errors  is  provided  during  the  scenario  by  color  coded 
messages  on  the  screen.  When  these  error  messages  occur  the  game  is  halted  for  several 
seconds.  Participants  were  given  score  sheets  on  which  they  put  a  check  mark  under  the 
appropriate  category  each  time  an  error  message  was  display^  (see  appendix  B,  page  B-6). 

A  summary  error  count  for  separation  errors,  handoff  errors,  and  missed  approaches  is  also 
provided  by  TRACON  at  the  end  of  each  session.  Participants  were  to  write  out  these  total 
error  counts  for  each  of  the  categories  at  the  end  of  a  session  to  double  check  their  own  error 
count  accuracy.  The  error-point  total  was  calculated  by  the  researchers  following  each 
session  of  testing. 

In  addition  to  the  error-point  total,  the  game  score  provided  by  TRACON  n  at  the  conclusion 
of  each  session  was  also  analyzed  as  a  performance  measure.  Higher  scores  reflected  better 
performance.  Since  the  analyses  conducted  on  scores  did  not  provide  any  additional 
information  to  that  obtained  from  analysis  of  the  error-point  totals,  they  are  not  described 
further. 

In  addition  to  working  under  the  test  conditions,  participants  also  filled  out  information  on  a 
post-session  questionnaire  after  each  session,  which  provided  subjective  ratings  of  their 
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performance  (see  appendix  B,  pages  B-7  through  B-9).  Participants  indicated  their  level  of 
agreement  with  each  probe  question  by  way  of  a  10-point  rating  scale  (12  in  the  case  of 
workload).  This  questionnaire  probed  for  factors  that  may  have  contributed  to  performance 
during  the  session  (e.g.,  fatigue,  stress).  ANOVA’s  were  conducted  on  these  post-session 
questionnaire  variables  to  determine  whether  they  differed  as  a  function  of  test  condition. 
The  questionnaire  also  allowed  for  open  responses  to  questions  concerning  individual 
strategies.  This  information  was  also  examined  to  determine  whether  certain  self-reported 
strategies  correlated  with  performance. 

The  post-session  questionnaire  variables  and  the  preliminary  questionnaire  variables  were 
also  analyzed  by  multiple  regression  analyses  to  obtain  partial  correlations  between  each 
variable  and  performance.  These  were  obtained  for  preliminary  questionnaire  variables  to 
determine  whether  performance  in  the  experiment  was  correlate  with  the  number  of 
semesters  completed,  number  of  ground  courses  taken,  number  of  flight  hours  completed, 
computer  experience,  video  game  experience,  and  stress  level.  These  were  obtained  for 
post-session  questionnaire  variables  to  determine  whether  performance  was  correlated  with 
self-reports  about  performance  after  each  session.  These  included  self-assessment  reports 
and  reports  about  perceived  levels  of  workload,  thinking  and  concentration,  busyness,  stress, 
and  fatigue. 

3.6  RESULTS. 

Overall  for  Experiment  1,  error-point  totals  averaged  4.96  and  were  very  variable 
(S.D.  =4.84).  Eight  of  the  participants  made  no  errors  in  at  least  one  of  the  two  test 
conditions.  One  participant  made  no  errors  in  either  condition.  Others,  however,  made  a 
number  of  errors  under  both  conditions.  The  highest  error-point  total  for  any  participant  in 
this  experiment  was  14. 

The  mean  error-point  total  was  lower  than  that  predicted  prior  to  the  start  of  the  experiment. 
Preliminary  estimates  had  been  arrived  at  by  evaluating  the  performance  of  two  novice 
TRACON  users  prior  to  the  work  conducted  with  the  group  at  MCCC.  Scenarios  for  the 
experiment  were  then  programmed  based  on  these  preliminary  results  that  indicated  what 
performance  level  was  likely  to  be. 

The  ANOVA  indicated  that  there  was  no  main  effect  of  test  condition  on  performance. 

Mean  error-point  totals  did  not  differ  significantly  between  the  PLANNING  and  NO¬ 
PLANNING  conditions,  F(l,ll)  =  .72,  p>.l  (see  table  3-2). 


14 


TABLE  3-2.  MEAN  ERROR-POINT  TOTAL  (AND  STANDARD 
DEVIATION)  FOR  EACH  TEST  CONDITION  IN 
EXPERIMENT  1 


PLANNING 

NO-PLANNING  I 

5.77  (4.97) 

4.15  (4.71)  1 

However,  analyses  of  the  post-session  questionnaire  variables  indicated  that  subjectively, 
participants  tended  to  report  that  their  thinking  and  concentration  level  was  somewhat  higher 
when  they  worked  under  the  NO-PLANNING  than  the  PLANNING  condition.  ANOVA’s 
conducted  on  these  data  revealed  that  this  was  the  only  post-session  questionnaire  variable 
whose  difference  approached  significance  (F(l,ll)=4.45,  p<  .1).  Means  and  standard 
deviations  for  all  post-session  questionnaire  items  for  the  PLANNING  and  NO-PLANNING 
conditions  are  shown  in  table  3-3. 

TABLE  3-3.  MEAN  RATING  AND  STANDARD  DEVIATION  FOR 
EACH  POST-SESSION  QUESTIONNAIRE  VARIABLE 
AND  EACH  TEST  CONDITION  IN  EXPERIMENT  1 


PLANNING 

NO-PLANNING  || 

Mean 

SD 

Mean 

SQ 

WorMoad 

5.00 

2.42 

5.39 

1.98 

Self-assessment 

7.54 

1.76 

7.85 

2.38 

Busyness 

6.69 

2.84 

6.15 

2.76 

lliiiiking  and  Concentration 

5.85 

2.91 

6.39 

2.60 

Stress 

4.85 

2.97 

5.46 

2.82 

Fatigue 

3.69 

2.81 

3.62 

2.73 

The  main  effect  of  ORDER  was  also  investigated.  It  was  possible  that  the  order  in  which  the 
participants  worked  affected  their  performance.  Six  of  the  participants  were  tested  under  the 
PLANNING  condition  on  the  first  day  of  testing  and  the  NO-PLANNING  condition  on  the 
second  day  (ORDER  =1),  while  the  other  seven  were  tested  under  the  NO-PLANNING 
condition  on  the  first  day  and  PLANNING  on  the  second  (ORDER=2).  Participants  who 
worked  under  the  PLAI^NG  condition  on  the  first  day  may,  for  example,  have  had  a  carry 
over  effect  to  the  second  day  of  testing  which  improved  performance  under  the  NO¬ 
PLANNING  condition.  This  group  may  have  been  better  prepared  to  use  minimal  advance 
information  such  as  that  provided  in  the  NO-PLANNING  condition  since  they  had  already 
had  experience  working  on  the  development  of  planning  strategies.  Those  who  worked  under 
the  NO-PLANNING  condition  on  the  first  day  would  be  less  likely  to  show  such  a  carry 
over  in  performance.  Means  for  each  PLAN  and  ORDER  condition  are  shown  in  flgure  3-1. 
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plan 


no  plan 


condition 

nCURE  3-1.  MEAN  ERROR-POINT  TOTAL  AND  STANDARD  ERROR  FOR 

EACH  TEST  CONDITION  AND  EACH  GROUP  IN  EXPERIMENT  1 
(ORDERS  1:  Group  that  performed  under  PLANNING  condition  Hrst 
ORDERs2:  Group  that  performed  under  NO-PLANNING  condition 
Tirst) 


The  ANOVA  revealed  that  the  main  effect  of  ORDER  did  not  reach  significance, 

F(1 , 1 1)  =2. 10,  p > .  1 .  Neither  was  the  effect  of  the  PLAN  x  ORDER  interaction  significant, 
F(l,ll)=.85,  p>  .1.  However,  an  analysis  of  covariance  (ANCOVA)  conducted  on  these 
data  did  indicate  that  these  groups  tended  to  differ  in  their  performance  under  the  NO- 
PLANNING  condition,  F(l,10)=3.19,  p<.l.  In  this  analysis,  performance  under  the 
PLANNING  condition  was  held  constant  while  error-point  totals  between  the  groups  were 
compared  for  the  NO  PLANNING  condition  alone.  In  addition,  the  group  that  worked  under 
the  PLANNING  condition  on  the  first  day  had  a  mean  error-point  total  of  5.50  and  a  mean 
error-point  total  of  only  1.83  when  they  worked  under  the  NO-PLANNING  condition  on  the 
second  day.  Although  a  repeated  measures  t-test,  conducted  to  compare  performance  under 
the  two  conditions  for  this  subgroup,  did  not  reach  significance,  t(5)=  +1.84,  p>  .1,  five  of 
these  six  participants  had  improved  performance  between  the  first  and  second  day.  The 
remaining  participant  did  not  make  any  errors  on  the  first  day  (PLANNING)  and  made  only 
one  error  on  the  second  day  (NO-PLANNING).  This  was  a  separation  conflict  which 
accounted  for  5  error  points. 

For  the  group  of  participants  that  worked  under  the  NO-PLANNING  condition  on  the  first 
day  and  the  PLANNING  condition  on  the  second  day,  this  trend  was  not  apparent.  These 
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participants  averaged  6.14  error  points  the  first  day  and  6.00  error  points  on  the  second. 

Only  two  of  these  seven  participants  showed  an  improvement  from  the  first  day  (NO¬ 
PLANNING)  to  the  second  (PLANNING).  One  of  the  other  five  participants  made  no  errors 
in  either  condition. 

Since  these  two  groups  of  participants  demonstrated  that  their  performance  tended  to  differ 
somewhat,  ANOVA’s  were  conducted  to  determine  whether  they  differed  in  terms  of  their 
responses  on  the  preliminary  questionnaire.  Means  and  standard  deviations  for  preliminary 
questionnaire  variables  for  each  group  are  shown  in  table  3-4.  None  of  these  differences 
reached  significance  (p  > .  1). 

TABLE  3-4.  MEAN  RATING  AND  STANDARD  DEVIATION  FOR  EACH 
PRELIMINARY  QUESTIONNAIRE  VARIABLE  AND  EACH 
TEST  CONDITION  IN  EXPERIMENT  1 


ORDER  = 1 

PLANNING  First 

NO-PLANNING  First  || 

Mean 

SD 

Mean 

SD 

Flight  Hours 

120.67 

88.04 

85.43 

91.79 

Semesters  Completed 

3.00 

1.79 

2.29 

1.11 

Ground  Courses  Completed 

3.33 

3.32 

2.43 

1.72 

Computer  Experience 

5.00 

2.37 

5.00 

1.83 

Video  Game  Experience 

8.17 

.75 

7.57 

2.07 

Stress 

3.16 

2.14 

5.14 

2.61 

Vision 

8.50 

1.97 

9.14 

1.86 

Volunteer 

10.00 

0 

9.71 

.98 

Health 

9.67 

.82 

9.43 

.98 

Motivation 

1.67 

1.21 

1.74 

1.89 

3.6.1  Post-hoc  Analyses. 

Performance  levels  varied  among  participants  regardless  of  experimental  condition.  The 
median  raw  error  totals  summed  over  both  conditions  (PLANNING  +  NO-PLANNING)  was 
2.  Eight  of  the  participants  made  between  zero  and  two  errors  combined  in  the  PLANNING 
and  NO  PLANNING  conditions  (mean  =  1.25,  SD=.71).  Five  participants  made  over  two 
errors  in  both  conditions  (mean =5.6,  SD=2.5).  Post-hoc  analyses  were  conducted  on  these 
two  groups  to  investigate  the  potential  causes  of  the  variability  in  observed  performance. 

Variables  measured  by  the  preliminary  questionnaire  indicated  that  there  was  a  significant 
difference  in  the  level  of  stress  each  reported  before  entering  the  experiment.  An  ANOVA 
conducted  on  stress  level  with  performance  as  a  factor  indicated  that  participants  with  fewer 
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errors  reported  a  significantly  lower  level  of  stress  prior  to  participating  in  the  experiment 
(mean=3.13,  SD  =  1.89)  than  did  those  who  made  more  errors  (mean=6.00,  SD=2.55), 

F(l,  11)  =5.499,  p<  .05,  There  was  also  a  small  difference  in  the  mean  number  of  semesters 
each  group  had  completed  prior  to  the  experiment.  Participants  with  two  or  fewer  errors, 
reported  completing  an  average  of  3.13  semesters  (SD=1.55),  while  those  with  more  than 
two  errors  reported  completing  an  average  of  only  1.8  semesters  (SD  =  .84).  The  ANOVA, 
however,  indicated  that  this  difference  just  missed  significance,  F(l,ll)=3.02,  p>.l. 

Regardless  of  category,  performance  was  significantly  correlated  with  reported  stress  level, 
the  number  of  ground  courses  taken,  and  the  level  of  video  game  experience  indicated  prior 
to  the  experiment  (p<  .05).  The  correlation  matrix  is  presented  in  table  3-5.  To  test  this,  a 
multiple  regression  analysis  was  conducted  on  all  preliminary  questionnaire  variables  with 
error-point  total  as  the  dependent  variable.  This  procedure  examined  the  effect  of  each 
variable  alone  after  partitioning  out  the  effect  that  it  had  in  combination  with  any  of  the 
others.  These  partial  correlations  indicated  that  an  increase  in  error-point  total  was  positively 
correlated  with  reported  stress  level  (•F.93)  and  video  game  experience  (4-. 74),  while 
negatively  correlated  with  the  number  of  ground  courses  taken  (-.84).  In  other  words, 
poorer  performers  indicated  higher  levels  of  stress  and  fewer  aviation  courses,  as  well  as  a 
higher  level  of  video  game  experience.  None  of  the  other  preliminary  questionnaire 
variables  reached  significance. 

TABLE  3-5.  CORRELATION  MATRIX  FOR  ALL  PRELIMINARY 

QUESTIONNAIRE  VARIABLES  AND  PERFORMANCE 


WT 

AGE 

SEM 

GRD 

FLT 

CO 

VID 

VIS 

STR 

MO 

WT 

1.00 

.39 

-.29 

-.36 

-.28 

.06 

.003 

-.51 

.77 

.48 

AGE 

.39 

1.00 

.11 

.15 

.51 

-.50 

.10 

-.46 

.53 

.40 

SEM 

-.29 

.11 

1.00 

.64 

.46 

-.01 

.12 

.35 

.04 

-.35 

GRD 

-.36 

.15 

.64 

1.00 

.82 

-.13 

.39 

.09 

.04 

-.04 

FLT 

-.28 

.51 

.46 

.83 

1.00 

-.36 

.56 

.09 

.05 

-.03 

CO 

.06 

-.50 

-.01 

-.13 

-.36 

1.00 

-.09 

,28 

-.04 

-.03 

VID 

.003 

.10 

.12 

.39 

.56 

-.09 

1.00 

.37 

-.22 

-.37 

VIS 

-.51 

-.49 

.35 

.09 

.09 

.28 

.37 

1.00 

-.54 

-.91 

STR 

.77 

.53 

.04 

.04 

.05 

-.04 

-.22 

-.54 

1.00 

.57 

MO 

.48 

.40 

-.35 

-.04 

-.03 

-.03 

-.37 

-.91 

.57 

1.00 

Note:  WT=total  error-point  score,  AGE = participant’s  age,  SEM^semesters  in  program,  GRD=number  of 
courses  taken,  FLT=number  of  flight  hours,  CO=:computer  experience,  VlD^video  game  experience, 
VIS=vision,  STR=stress,  MO = motivation. 
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Analyses  of  the  post-session  questionnaire  variables  also  reflected  some  differences  between 
the  group  that  made  fewer  errors  and  the  group  that  made  more  errors.  ANOVA’s  were 
conducted  on  each  of  the  post-session  variables  with  performance  as  a  factor.  These  groups 
differed  significantly  in  their  workload  assessments,  F(l,ll)=7.39,  p<.05.  The  group  with 
fewer  errors  had  a  mean  workload  assessment  rating  of  4.19  (SD=1.6)  while  the  group  with 
more  errors  had  a  rating  of  6.80  (SD=2.08).  The  group  with  fewer  errors  reported  a  lower 
level  of  busyness  for  the  sessions  (mean=5.31,  SD=2,59)  than  did  the  group  with  more 
errors  (mean=8.2,  SD=2.27),  F(l,ll)=4.68,  p<  .05.  The  group  with  fewer  errors  also 
reported  a  lower  level  of  stress  during  each  session  (mean=3.69,  SD=2.11)  than  the  group 
with  more  errors  (mean=7.5,  SD=2.23),  F(l,ll)=11.32,  p=.01. 

In  addition,  the  group  that  made  fewer  errors  reported  a  somewhat  higher  self-assessment 
rating  (mean=8.19,  SD=1.56)  than  did  those  with  more  errors  (mean=6.9,  SD=2.47), 
F(l,ll)=3.7,  p<  .1.  The  group  that  made  fewer  errors  also  reported  a  lower  level  of 
fatigue  following  the  session  (mean=2.31,  SD=1.56)  than  the  group  that  made  more  errors 
(mean=5.8,  SD=3.01),  F(l,ll)=8.99,  p<.01. 

Regardless  of  category,  performance  was  significantly  correlated  with  perceived  workload. 

A  multiple  regression  analysis  was  conducted  on  all  post-session  questionnaire  variables  with 
error-point  total  as  the  dependent  variable.  The  correlation  matrix  is  shown  in  table  3-6. 

This  analysis  examined  the  relationship  between  overall  post-session  assessments  (e.g., 
overall  workload = workload  [PLANNING]  -I-  workload  [NO-PLANNING] )  and 
performance.  Partial  correlations  determined  the  relationship  between  each  variable  alone 
and  performance  after  partitioning  out  the  relationship  that  each  had  with  others.  Error-point 
total  was  positively  correlated  with  workload  (+.78,  p<.05).  That  is,  perceived  workload 
levels  increased  as  errors  increased.  None  of  the  other  post-session  questionnaire  variables 
reached  significance. 

TABLE  3-6.  CORRELATION  MATRIX  FOR  ALL  POST-SESSION 
QUESTIONNAIRE  VARIABLES  AND  PERFORMANCE 


WT 

SELF 

THINK 

TIRED 

WORK 

BUSY 

STR 

WT 

1.00 

-.52 

.42 

.68 

.78 

.57 

.77 

SELF 

-.52 

1.00 

-.65 

-.71 

-.64 

-.57 

-.59 

THINK 

.42 

-.65 

1.00 

.47 

.78 

.85 

.75 

1^^91111 

.68 

-.71 

.47 

1.00 

.73 

.55 

.75 

WORK 

.78 

-.64 

.78 

.73 

1.00 

.86 

.87 

BUSY 

.57 

-.57 

.85 

.55 

.86 

1.00 

.79 

STR 

.77 

-.59 

.75 

.75 

.87 

.79 

1.00 

Note:  WT=total  error-point  score,  SELF=seIf  assessment  rating,  'IHlNK=thinking  and  concentration, 
TIRED = fatigue,  WORK = workload,  BUSY = busyness,  STR=stress. 
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As  an  additional  part  of  the  post-hoc  analyses,  the  strategies  reported  by  participants  in  these 
two  groups  were  compared.  It  may  have  been  possible  that  subjects  in  the  group  with  better 
overall  performance  were  using  more  effective  strategies  than  participants  who  performed 
more  poorly.  While  there  was  no  way  to  determine  what  strategies  were  used  during  the 
course  of  the  session,  participants  were  asked  to  report  any  strategies  they  used  in  the  post¬ 
session  questionnaire  after  the  session  was  complete.  These  reports  were  then  examined  for 
indications  that  some  type  of  specific  strategy  had  been  used  (e.g.,  "descended  arrivals 
immediately  to  approach  fix"). 

Seven  of  the  eight  participants  from  the  group  that  made  fewer  errors  reported  specific 
strategies  while  only  two  of  the  five  participants  in  the  group  that  made  a  higher  number  of 
errors  did  so  (see  table  3-7  for  samples  of  participant’s  reports).  The  other  participants  did 
not  provide  evidence  in  their  reports  that  they  used  a  specific  strategy  (e.g.,  "uncomfortable 
not  being  able  to  see  pending  strips").  The  trends  appear  to  be  in  the  direction  expected  if 
strategies  assist  performance,  however,  these  differences  did  not  reach  significance.  A  chi- 
square  test  of  independence  indicated  that  the  number  of  participants  responding  with  either 
specific  or  nonspecific  strategies  did  not  differ  between  these  groups,  chi-square(l)  =  1.42, 
p>.l. 


TABLE  3-7.  SAMPLES  OF  REPORTED  STRATEGIES  FROM  POST-SESSION 
QUESTIONNAIRE  IN  EXPERIMENT  1 


Specific 

Nonspecific 

"Landing:  turn  aircraft  as  soon  as  possible 
onto  a  heading  that  will  set  them  up  for 
final  approach  and  then  slow  them  down 
and  descend  them." 

"...with  flight  strips...!  was  able  to  prepare 
myself  as  to  the  expedition  of  each 
aircraft, " 

"Have  traffic  immediately  go  to  the  altitude 
it  would  eventually  have  to  be  at  for 
approach.  I  would  tell  the  aircraft  to  go 
directly  to  the  airport  when  it  was  inside 
the  approach  path  instead  of  giving  it 
vectors." 

"Using  the  flight  strips  helped  to  see  where 
the  aircraft  had  to  go  and  made  it  much 
easier  to  control." 

"Vectored  plane  directly  to  a  final  approach 
position  for  their  airports  as  planes  came 
in.  With  overflights  I  just  gave  them  radar 
contact  and  didn’t  communicate  with  them 
until  the  handoff." 

"Looked  ahead  in  time  about  5  minutes." 
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3.7  DISCUSSION. 


The  results  of  Experiment  1  did  not  reveal  a  difference  in  performance  between  a  condition 
designed  to  encourage  planning  and  strategy  development  and  one  designed  to  discourage 
these  activities.  A  possible  reason  for  this  outcome  may  have  been  that  these  experimental 
conditions  did  not  alter  the  opportunity  to  plan  as  intended.  Flight  strip  removal,  which  was 
intended  to  hinder  the  ability  to  form  plans  before  the  session  began,  may  not  have 
eliminated  the  opportunity  to  plan  entirely.  Planning  may  have  been  carried  out  successfully 
after  the  session  had  gotten  underway.  Scenarios  may  not  have  been  challenging  enough  to 
have  made  preplanning  necessary  to  successful  performance.  Based  on  the  number  of 
participants  who  were  able  to  work  through  these  sessions  with  minimal  errors,  this  seems  a 
likely  possibility.  Several  of  the  written  and  verbal  comments  made  by  participants  after  the 
experiment  lend  some  support  to  this  view.  While  participants  acknowl^ged  feeling  more 
"comfortable"  having  the  paper  flight  strips  available  and  also  indicated  requiring  a  lower 
level  of  thinking  and  concentration  for  the  PLANNING  condition  when  flight  strips  were 
available,  many  indicated  that  they  felt  competent  enough  to  have  been  able  to  work  through 
the  selected  scenarios  without  them. 

Participants  varied  widely  in  their  overall  performance  levels.  Error-point  totals  ranged  from 
0  to  14.  Eight  participants  (62  percent)  made  no  errors  when  working  under  either  one  or 
both  conditions,  thereby  making  the  effect  of  even  one  error  in  the  alternate  condition 
substantial.  The  tendency  toward  such  a  low  number  of  errors  for  so  many  participants  was 
not  expected.  One  possible  reason  for  the  low  error  rate  may  have  been  the  addition  of  the 
two  training  sessions,  one  of  which  allowed  participants  practice  with  the  same  number  of 
aircraft  that  were  presented  in  the  experiment.  This  practice  may  have  enhanced 
performance  for  some  to  an  almost  perfect  level. 

Other  factors  were  investigated  to  determine  likely  reasons  for  performance  variability.  One 
may  have  been  the  level  of  stress  that  participants  acknowledge  before  entering  the 
experiment.  Those  who  made  fewer  errors  indicated  a  somewhat  lower  level  of  stress  than 
did  those  who  made  more.  Aviation  experience  may  also  have  influenced  performance.  The 
number  of  ground  courses  completed  was  significantly  correlated  with  errors.  Error  scores 
decreased  with  the  number  of  courses  completed. 

Reactions  to  performance  in  the  experiment  also  differed  between  the  group  of  participants 
that  made  fewer  errors  and  those  that  made  more.  The  group  that  made  fewer  errors 
indicated  that  their  level  of  workload,  busyness,  and  stress  were  lower  than  those  for  the 
group  that  made  more  errors.  The  group  that  made  fewer  errors  also  reported  a  lower  level 
of  fatigue  and  assessed  their  performance  more  highly.  Additionally,  partial  correlations  of 
post-session  questionnaire  variables  and  performance  indicated  that  workload  correlated 
significantly  and  positively  with  error-point  total.  Perceived  workload  levels  increased  with 
errors. 
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Performance  variability  may  also  have  been  due  to  variations  in  the  effectiveness  of 
individual  strategies.  Participants  in  this  experiment  were  not  explicitly  instructed  to  plan 
specific  strategies.  The  intent  of  the  experiment  was  only  to  measure  performance  when  the 
opportunity  for  planning  was  made  available.  Some  options  and  suggestions  were  given 
(e.g.,  sketch  the  flightpath),  but  beyond  that,  individu^s  developed  and  implemented  their 
own  specific  plans.  Some  may  have  developed  very  effective  strategies  while  others  did  not. 
Or,  some  may  have  found  strategies  easier  to  develop  than  others.  Some  participants,  for 
example,  may  not  have  been  able  to  quickly  identify  and  implement  an  appropriate 
organizational  scheme.  Such  individual  differences  in  the  overall  ability  to  generate  and 
execute  helpful  plans  may  have  resulted  in  some  of  the  differences  observed  between 
participants’  performance  levels  in  this  experiment.  These  individually  imposed  strategies 
may  have  played  a  strong  role  in  performance  which  outweighed  the  experimentally  imposed 
conditions. 

Bousfield’s  (1953)  work  demonstrated  that  experimental  subjects  who  imposed  their  own 
strategies  enhanced  their  performance  on  a  re^l  task.  In  his  experiment,  participants  were 
presented  a  list  of  words  each  of  which  belonged  to  one  of  four  different  categories  (names, 
animals,  vegetables,  and  professions).  The  words  were  presented  randomly,  but  were 
recalled  by  category.  This  type  of  strategy  has  been  observed  to  increase  the  number  of 
words  remembered  relative  to  a  condition  in  which  no  organizational  scheme  is  used. 
Additional  work  by  Bransford  and  Johnson  (1972,  1973)  has  also  indicated  that 
organizational  strategies  improve  memory.  They  found  that  participants  in  their  experiments 
who  made  up  their  own  context  while  hearing  an  ambiguous  passage,  remembered  more  of 
the  passage  than  those  who  claimed  not  to  have  used  a  context. 

Although  it  was  not  possible  to  determine  much  about  the  individual  strategies  used  by 
participants  in  this  experiment,  some  information  could  be  obtained  from  the  post-session 
questionnaire  which  specifically  asked  what  strategies  participants  used  during  the  session 
they  had  just  completed.  To  investigate  strategies  more  directly,  participants  would  have  had 
to  have  been  observed  more  closely  while  working  through  a  session,  explaining  how  and 
why  they  were  directing  traffic  as  they  were.  This  information  could  then  be  evaluated  by 
protocol  analysis.  While  the  design  of  this  experiment  did  not  allow  for  this,  some  reports 
from  the  post-session  questionnaire  were  revealing.  Most  of  the  participants  who  made  the 
fewest  number  of  errors  reported  specific  strategies  for  the  way  in  which  they  handled  traffic 
in  either  one  or  both  conditions.  Most  focused  on  arrival  aircraft  which  are  the  most 
difficult  to  manage.  Several  participants  indicated,  for  example,  that  they  took  steps  to  set 
up  aircraft  early  for  final  approach.  That  is,  as  soon  as  contact  was  made  with  an  aircraft 
that  was  to  land,  they  descended  the  aircraft  immediately  to  final  altitude  and  vectored  it  as 
soon  as  possible  to  final  approach  heading.  Verbal  comments  made  after  the  experiment’s 
completion  by  participants  using  this  strategy,  indicated  that  they  chose  to  do  this  by  sending 
aircraft  direct  to  the  fix  located  just  outside  the  final  approach  path  of  an  airport,  then  turned 
the  aircraft  onto  final  approach  and  handed  it  off  to  land.  This  acts  to  standardize  the 
process  for  landing.  Such  a  strategy  reduces  the  number  of  commands  that  need  to  be  issued 
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to  an  aircraft  and  helps  organize  the  procedure.  Doing  this,  in  turn,  presumably  leaves  more 
time  available  for  controllers  to  handle  other  aircraft  or  to  search  for  potential  conflicts. 

4.  EXPERIMENT  2:  EFFECT  OF  PLANNING  TIME  ON  PERFORMANCE. 

4.1  PURPOSE. 

The  purpose  of  the  second  experiment  was  to  investigate  how  the  amount  of  available 
planning  time  affects  the  ability  to  control  aircraft.  It  has  been  indicated  that  a  large  number 
of  controller  errors  are  made  during  the  initial  IS  minutes  after  a  controller  has  taken  over  a 
position  (Vingelis  et  al.,  1990).  These  errors  may  result  because  the  controller  has  not 
allowed  for  sufficient  planning  time  to  set  up  the  appropriate  "picture"  of  the  situation  in 
which  he/she  is  suddenly  immersed.  Allowing  for  more  planning  time  should  enhance 
performance  relative  to  a  condition  which  allows  for  less  time  to  plan. 

4.2  METHOD. 

4.2.1  Participants. 

Twelve  of  the  students  who  participated  in  Experiment  1  participated  in  Experiment  2.  Data 
from  the  preliminary  questionnaire,  therefore,  were  about  the  same  as  those  for  Experiment 
1.  The  ages  of  the  participants  ranged  from  18  to  26  (mean =20.67,  SD =2.57).  They  had 
completed  from  one  to  five  semesters  in  the  aviation  program  (mean =2.67,  SD=1.5)  and 
from  one  to  eight  basic  ground  courses  (mean  =3. 18,  SD=2.56).  Their  flight  experience 
varied  from  0  to  250  hours  completed  prior  to  the  start  of  the  experiments  (mean= 105.5, 
SD=90.98).  Two  participants  indicate  having  had  some  aviation  experience  prior  to 
attending  Mercer  County  Community  College,  but  did  not  elaborate  on  what  that  involved. 

Participants’  ratings  (l=lowest,  10=highest)  on  their  level  of  computer  experience  ranged 
from  2  to  8  (mean =5.08,  SD=2.07),  video  game  experience  ranged  from  6  to  10 
(mean=8.08,  SD=1.38),  and  quality  of  vision  ranged  from  5  to  10  (mean=8.75, 

SD=1.91).  None  indicated  any  color  vision  deticiency.  Participants  indicated  high 
agreement  in  their  willingness  to  participate  in  the  study  (mean =9. 83,  SD=.58).  They  also 
indicated  high  ratings  for  their  general  health  (mean =9.50,  SD=.90).  They  did  indicate 
variable  levels  of  recent  stress  (mean =4.33,  SD=2.61).  None  indicated  that  they  were 
taking  any  medication  that  would  interfere  with  their  mental  or  motor  abilities.  Level  of 
motivation  to  participate  was  generally  high  (mean =1.75,  SD=1.60).  This  question  was 
worded  so  that  1  was  equal  to  the  highest  level  of  agreement,  10  the  lowest. 

4.2.2  Test  Conditions. 

Two  experimental  conditions  were  tested  that  were  designed  to  simulate  conditions 
experienced  by  controllers  taking  over  a  position.  In  both  conditions  participants  were 
informed  that  they  would  view  an  ongoing  scenario  and,  that  at  some  point,  they  would  be 
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required  to  tate  over  control  of  the  aircraft.  They  were  instructed  that  the  experimenter 
would  indicate  verbally  when  to  take  control.  One  condition  mimicked  a  situation  in  which 
Uttle  planning  time  was  allotted  before  control  of  the  airspace  was  assumed.  The  second 
condition  mimicked  a  situation  in  which  increased  planning  time  was  allotted.  Each 
participant  was  tested  under  each  condition  as  described  below. 

4.2.2. 1  2-MINUTE  condition. 

Under  this  condition,  participants  were  tested  on  their  performance  controlling  aircraft  after 
viewing  an  ongoing  scenario  for  2  minutes.  The  test  scenario  was  initially  controlled  by  one- 
half  of  the  participants  while  the  other  half  (test  group)  were  seated  across  the  room  so  that 
they  were  unable  to  watch  the  scene  and  the  way  that  the  traffic  was  being  handled.  The  test 
group  participants  were  instructed  to  join  their  partners  and  to  begin  viewing  the  scene  after 
a  signal  was  given.  They  were  told  to  be  prepared  to  take  control  of  the  scenario  as  soon  as 
the  words  "take  over"  were  announced  by  one  of  the  experimenters.  Participants  were  not 
told  how  much  time  they  wou^d  have  until  that  signal  was  given.  They  were  instructed  to 
use  the  available  time  to  observe  their  partners  working  on  the  ongoing  scenario  and  to  ask 
questions  in  order  to  prepare  themselves  in  ways  that  they  could  foresee  would  best  help 
them  direct  traffic  once  they  assumed  control.  Once  the  test  group  assumed  control,  they 
worked  on  the  scenario  for  the  next  IS  minutes  while  recording  their  errors  on  a  response 
sheet  as  described  in  Experiment  1. 

4.2.2.2  5-MlNUTE  condition. 

Under  this  condition,  participants  were  tested  on  their  performance  controlling  aircraft  after 
viewing  an  ongoing  scenario  for  5  minutes.  The  test  scenario  was  initially  controlled  by  one- 
half  of  the  participants  while  the  other  half  (test  group)  were  seated  across  the  room.  All 
other  aspects  of  this  condition  are  the  same  as  those  described  for  the  2-  MINUTE  condition, 
except  that  the  test  group  was  given  5  minutes  to  observe  the  ongoing  scenario  before  being 
given  the  take  over  signal. 

4.3  DESIGN. 

As  in  Experiment  1,  participants  were  tested  according  to  a  2  x  2  mixed  design  with  two 
levels  of  each  factor  (see  table  4-1).  The  2-MINUTE  and  5-MINUTE  conditions  served  as 
the  two  levels  of  the  within-subjects  factor  (TIME).  Students,  therefore,  participated  under 
both  of  these  conditions.  The  order  in  which  the  participants  worked  under  each  condition 
was  the  between-subjects  factor  (ORDER).  One-half  of  the  participants  worked  under  the  2- 
MINUTE  condition  on  the  first  day  of  testing  and  one-half  worked  under  the  5-MINUTE 
condition.  On  the  second  day,  participants  worked  under  the  alternate  condition.  Main 
effects  of  TIME  and  ORDER  as  well  as  the  effect  of  their  interaction  were  analyzed  by  an 
ANOVA  using  SPSS. 
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TABLE  4-1.  DESIGN  FOR  EXPERIMENT  2  (Partidpaiits  Worked  Under 
Both  the  2-MINUTE  and  5-MINUTE  Conditions  in 
the  Order  Indicated) 


1 - 

2-MINUTE 

5-MINUTE 

1  ORDER  =  1 

1st 

2nd 

2nd 

1st 

Two  programmed  scenarios  were  developed  as  described  in  the  Equipment  section  above. 
These  scenarios  were  the  same  except  for  the  call  letters  of  the  aircraft.  Scraarios  included 
26  aircraft  presented  within  a  50-minute  time  span.  Longer  scenarios  were  needed  than  those 
in  Experiment  1  since  participants  alternated  control  of  the  scenario  on  each  day  of  testing. 
One  scenario  was  used  for  each  day. 

Test  scenarios  were  revised  from  those  originally  developed  for  this  experiment  based  on  the 
observed  level  of  performance  in  Experiment  1.  In  that  experiment,  8  of  the  13  participants 
completed  at  least  one  test  session  without  making  any  errors.  Since  error  scores  served  as 
the  dependent  variable,  it  was  determined  that  scenarios  should  be  increased  in  complexity  in 
order  to  obtain  a  greater  number  of  errors.  More  aircraft  per  unit  time  and  more  built-in 
conflicts  were,  therefore,  added  to  the  programmed  scenarios. 

The  dependent  variable  was  the  error-point  total  described  in  Experiment  1.  As  in  the  first 
experiment,  participants  were  to  record  their  errors  on  a  response  sheet  as  the  session 
progressed.  Since  participants  were  entering  and  exiting  the  scenario  at  different  times, 
summary  performance  information  could  not  be  obtained  from  TRACON  n.  This  summary 
information  can  only  be  obtained  when  the  scenario  is  ended  or  temporarily  "closed." 

Neither  of  these  options  were  viable  given  that  ongoing  scenarios  were  necessary  in  order  for 
participants  to  alternate  control  without  a  disruption  in  the  traffic  flow.  Errors  were  instead 
obtained  only  from  participant’s  response  sheets  which  were  totalled  by  the  experimenters 
after  completion  of  the  session. 

Participants  completed  the  post-session  questionnaite  at  the  end  of  each  test  session.  This 
information  was  analyzed  by  ANOVA’s  to  determine  whether  factors  such  as  stress, 
workload,  etc.,  differed  as  a  function  of  the  experimental  conditions. 

Multiple  regression  analyses  were  also  conducted  on  variables  obtained  from  the  preliminary 
and  post-session  questionnaires  to  determine  whether  performance  was  correlated  with  any  of 
these  factors 

4.4  PROCEDURE. 

Experiment  2  was  conducted  on  2  days.  Participants  worked  in  randomly-assigned  pairs. 

On  the  first  day,  one-half  of  the  participants  worked  under  the  2-MINUTE  viewing  condition 
first,  followed  by  the  other  half,  who  worked  under  the  S-MINUTE  viewing  condition. 
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At  the  beginning  of  the  session,  one  member  of  each  pair  began  the  scenario  and  worked  for 
5  minutes.  The  other  member  of  each  pair  was  seated  across  the  room  until  instructed  to 
rejoin  his/her  partner.  After  rejoining,  this  test  group  was  instructed  to  watch  their  partners 
work  so  that  they  would  be  ready  to  take  control  of  the  scenario  when  the  take  over  signal 
was  given.  This  signal  came  after  2  minutes  of  viewing.  The  test  group  then  took  control  of 
the  scenario  for  the  next  IS  minutes  and  monitored  their  performance  by  indicating  errors  on 
the  checklist.  Their  partners  moved  to  the  other  side  of  the  room  once  control  was 
relinquished.  After  IS  minutes,  this  group  was  given  the  same  instructions  as  described 
above,  but  were  allowed  S  minutes  of  viewing  time  before  taking  control  of  the  scenario. 

On  the  second  day  of  testing,  participants  worked  under  the  alternate  condition.  Those  who 
began  the  session  the  previous  day  and  subsequently  participated  under  the  S-MINUTE 
condition,  today  worked  under  the  2-MINUTE  condition.  Those  who  had  previously  worked 
under  the  2-MINUTE  condition,  today  worked  under  the  S-MINUTE  condition. 

4.S  RESULTS. 

Many  more  errors  were  made  in  this  experiment  than  Experiment  1,  despite  the  fact  that  test 
sessions  were  shorter.  The  mean  overall  error-point  score  was  IS. 33  (SD= 12.42)  for 
Experiment  2,  and  was  4.96  (SD=4.84)  for  Experiment  1.  A  greater  proportion  of  errors 
was  expected  for  Experiment  2  since  the  scenarios  were  programmed  to  be  more  difficult. 

Participant’s  post-session  questionnaire  responses  also  indicated  the  increased  difficulty  of 
Experiment  2.  An  ANOVA  was  conducted  to  compare  the  responses  given  after  Experiment 
2  to  those  given  after  Experiment  1  for  the  12  participants  who  completed  both.  All  of  the 
post-session  questionnaire  variables  in  Experiment  2,  with  the  exception  of  fatigue,  were 
significantly  different  (p<  .05)  from  those  reported  in  Experiment  1.  Fatigue  neared 
significance  at  p<  .1.  The  means  and  standard  deviations  for  each  experiment  and  each 
variable  are  presented  in  table  4-2. 

TABLE  4-2.  MEANS  AND  STANDARD  DEVIATIONS  OF  POST-SESSION 
QUESTIONNAIRE  VARIABLES  FOR  EXPERIMENT  1  AND 
EXPERIMENT  2 


EXPERDVfEIST  1 

EXPERIMENT  2  || 

Mean 

SD 

Mean 

SD 

tVorkload 

10.58 

4.30 

19.67 

3.80 

Self-Assessment 

15.17 

2.59 

11.25 

3.08 

Busyness 

12.75 

5.58 

19.00 

1.41 

Thinking  and 
Concentration 

12.17 

5.69 

18.33 

2.61 

Stress 

10.67 

5.50 

17.42 

2.47 

Fatigue 

7.59 

5.64 

10.17 

4.37  1 
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The  means  and  standard  deviations  were  calculated  for  each  experiment  by  obtaining  a  total 
score  for  each  variable  summed  over  both  test  conditions.  For  example,  each  participant’s 
workload  score  for  Experiment  2  was  obtained  by  adding  the  workload  score  i^m  both  the 
2-MINUTE  and  5-MINUTE  conditions. 

Means  for  each  of  the  2-MINUTE  and  5-MINUTE  conditions  are  presented  in  table  4-3. 

TABLE  4-3.  MEAN  ERROR-POINT  TOTAL  (AND  STANDARD  DEVIATION)  FOR 
EACH  TEST  CONDITION  IN  EXPERIMENT  2 


2-MINUrE 

S-MINIJIE  1 

13.33  (8.39) 

17.33  (16.45)  1 

An  ANOVA  revealed  no  significant  main  effect  of  TIME  on  performance,  F(l,10)  =  1.08, 
p>.l.  There  was  also  no  main  effect  of  ORDER  on  performance,  F(l,10)=1.58,  p>.l. 
Participants  who  performed  under  the  2-MINUTE  condition  first  and  the  5-MINUTE 
condition  second  did  not  differ  in  overall  performance  from  those  who  performed  under  the 
5-MINUTE  condition  first  and  the  2-MINUTE  condition  second.  The  interaction  between 
TIME  and  ORDER  was  also  not  significant,  F(l,10)=.61,  p>  .1.  Means  for  each  TIME 
and  ORDER  condition  are  shown  in  figure  4-1. 


- ORDER  =  2 

- ORDER  =  1 


condition 


nCURE  4-1.  MEAN  ERROR-POINT  TOTAL  AND  STANDARD  ERROR  FOR 

EACH  TEST  CONDITION  AND  EACH  GROUP  IN  EXPERIMENT  2 
(ORDER =1:  Group  that  performed  under  2-MINUTE  condition  flrst 
orders: 2:  Group  that  performed  under  5-MINUTE  condition  flrst) 
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One  possible  reason  why  performance  was  not  improved  after  the  5-MINUTE  viewing 
condition  may  have  been  due  to  the  fact  that  the  5-MINUTE  condition  was  always  tested 
later  in  the  scenario  than  the  2-MINUTE  condition.  Early  in  the  scenario,  traffic  was 
relatively  light  (up  to  8  aircraft),  whereas  later  in  the  scenario,  traffic  could  potentially  have 
become  quite  heavy  (up  to  11  to  16  aircraft).  The  increase  in  the  number  of  aircraft 
provides  more  of  an  opportunity  for  the  scenario  to  get  complicated,  leading  to  more  of  an 
opportunity  for  errors.  Since  the  number  of  aircraft  present  at  different  times  in  the  scenario 
was  not  measured,  it  was  not  possible  to  further  investigate  the  way  in  which  errors  were 
affected  by  this  variable  directly. 

An  ANOVA  was  conducted  on  preliminary  questionnaire  variables  as  a  function  of  ORDER 
to  determine  if  these  two  groups  of  participants  differed  from  one  another  in  terms  of  any  of 
the  characteristics  with  which  they  entered  the  experiments.  The  difference  between  the 
level  of  stress  reported  by  these  groups  was  somewhat  significant,  F(l,10)=4.0,  p<  .1.  The 
group  that  performed  under  the  2-MINUTE  condition  first  had  reported  a  somewhat  lower 
level  of  stress  prior  to  entering  the  experiments  (mean =3.00,  SD=1.26)  than  the  group  that 
performed  under  the  5-MINUTE  condition  first  (mean=5.67,  SD=3.01).  None  of  the  other 
differences  from  the  preliminary  questionnaire  reached  significance. 

ANOVA’s  were  also  conducted  on  post-session  questionnaire  variables  to  determine  whether 
any  of  these  differed  as  a  function  of  TIME  and  ORDER.  Means  and  standard  deviations  for 
these  variables  for  the  2-MINUTE  and  5-MINUTE  conditions  are  presented  in  table  4-4. 

TABLE  4-4.  MEAN  RATING  AND  STANDARD  DEVIATION  FOR  EACH 
POST-SESSION  QUESTIONNAIRE  VARIABLE  AND  EACH 
TEST  CONDITION  IN  EXPERIMENT  2 


2-MINUTE 

5-MINUTE 

Mean 

SD 

Mean 

SD 

Workload 

10.00 

2.13 

9.67 

1.97 

Self-Assessment 

5.58 

1.51 

5.67 

2.50 

Busyness 

9.50 

1.17 

9.50 

.80 

Thinking  and 
Concentration 

9.08 

1.78 

9.25 

.97 

Stress 

8.75 

1.77 

8.67 

1.23 

Fatigue 

5.58 

2.61 

4.58 

2.75 
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As  indicated  in  the  table,  means  for  workload,  busyness,  thinking  and  concentration,  and 
stress  assessments  were  quite  high  under  both  conditions.  Nearly  all  were  at  the  highest 
response  level  possible  (workload  differed  from  the  others  in  that  it  was  based  on  a  12  point 
rather  than  a  10  point  scale).  Differences  between  the  2-MINUTE  and  5-MINUTE 
conditions  did  not  reach  signiiicance  for  any  post-session  questionnaire  variables  (p>.l). 
Neither  were  there  any  significant  differences  for  any  of  these  variables  as  a  function  of 
ORDER  (p>  .1).  There  was  one  variable,  self-assessment,  for  which  the  interaction  of 
TIME  X  ORDER  did  reach  significance,  F(l,10)=5.69,  p<  .05.  It  is  not  clear  what  would 
explain  these  differences  in  assessments.  They  do  not  correspond  to  observed  performance 
levels. 

4.5.1  Post-hoc  Analyses. 

As  in  Experiment  1,  performance  levels  varied  among  individual  participants  regardless  of 
experimental  condition.  The  median  raw  error  totals  summed  over  both  conditions 
(2-MINUTE  +  5-MINUTE)  was  5.5.  Seven  of  the  participants  made  between  two  and  six 
total  errors  (mean=3.5,  SD=1.38).  The  other  five  participants  made  between  9  and  22  total 
errors  (mean=12.6,  SD=5.68).  This  gap  in  participants’  performance  was  used  to  separate 
participants  into  two  groups.  The  participants  in  each  of  these  groups  were  not  necessarily 
the  same  participants  who  made  up  the  groups  making  fewer/greater  errors  in  Experiment  1. 
Four  of  the  seven  participants  who  made  fewer  errors  in  this  experiment,  also  made  fewer 
errors  in  the  first.  Two  of  the  five  participants  who  made  more  errors  in  this  experiment 
also  made  more  in  the  first  (see  table  4-5). 


TABLE  4-5.  NUMBER  OF  PARTICIPANTS  FROM  EXPERIMENT  1  WHO 
REMAINED  IN  THE  SAME  POST-HOC  ANALYSIS  GROUP  IN 
EXPERIMENT  2 


EXPERIMENT 

EXPERIMENT 

1 

2 

Fewer 

8 

4 

Errors 

More 

5 

2 

Errors 

An  ANOVA  was  conducted  on  all  preliminary  and  post-session  questionnaire  variables 
between  these  groups  and  across  the  2-MINUTE  and  5-MINUTE  conditions.  Since  the 
groups  were  defin^  by  the  total  number  of  errors  made,  error-point  totals  necessarily 
differed  significantly  between  them.  Of  interest,  however,  was  that  the  effect  of  the  TIME  x 
GROUP  interaction  was  significant,  F(l,10)=7.19,  p<  .05.  This  interaction  is  shown  in 
figure  4-2.  These  trends  indicate  that  the  group  that  made  fewer  overall  errors,  tended  to 
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reduce  errors  between  the  2-MINUTE  and  5-MINUTE  test  conditions,  while  the  group  that 
made  more  overall  errors,  made  more  errors  under  the  5-MINUTE  condition  than  the  2- 
MINUTE  condition. 

An  ANCOVA  was  also  conducted  in  which  the  total  error  for  the  2-MINUTE  condition 
served  as  the  covariate,  so  that  performance  under  the  5-MINUTE  condition  alone  could  be 
compared  between  these  groups.  This  analysis  revealed  that  the  two  groups  differed 
significantly  in  their  performance  under  the  5-MINUTE  condition,  F(l,9)=10.05,  p<  .05. 
The  group  that  made  fewer  errors  overall  made  significantly  fewer  errors  in  the  5 -MINUTE 
condition  than  did  the  group  that  made  a  greater  number  of  errors.  These  results  suggest 
that  participants  who  made  fewer  errors  overall  were  less  affected  by  having  to  work  with  a 
greater  number  of  aircraft  than  those  who  performed  more  poorly  overall. 


— more 
ERRORS 

— *  FEWER 
ERRORS 


condi  t  icxi 

nGURE  4-2.  MEAN  ERROR-POINT  TOTAL  AND  STANDARD  ERROR  FOR 

EACH  TEST  CONDITION  AND  EACH  POST-HOC  PERFORMANCE 
GROUP  IN  EXPERIMENT  2  (Fewer  Errors:  Group  that  made  6  or 
fewer  errors  over  both  conditions.  More  Errors:  Group  that  made  9  or 
more  errors  over  both  conditions.) 

ANOVA’s  conducted  on  the  preliminary  questionnaire  variables  indicated  that  none  differed 
significantly  between  these  groups  (p>  .1).  A  multiple  regression  analysis  was  conducted  on 
all  preliminary  questionnaire  variables  with  enor-point  total  as  the  dependent  variable.  The 
correlation  matrix  is  shown  in  table  4-6.  This  an^ysis  allowed  the  correlation  of  each 
variable  with  the  error-point  total  to  be  determined  after  partitioning  out  the  effect  that  it  had 
in  combination  with  others.  These  partial  correlations  also  did  not  indicate  that  any  of  the 
preliminary  questionnaire  variables  correlated  significantly  with  performance  (p>  .1). 
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TABLE  4-6.  CORRELATION  MATRIX  OF  PRELIMINARY 

QUESTIONNAIRE  VARUBLES  AND  PERFORMANCE 


WT 

AGE 

SEM 

GRD 

FLT 

CO 

VID 

VIS 

STR 

MO 

WT 

1.00 

-.06 

1 

b 

.19 

-.01 

.35 

.14 

.03 

-.11 

-.09 

AGE 

-.06 

1.00 

.09 

.13 

.50 

-.54 

-.002 

-.44 

.54 

.39 

SEM 

-.05 

.10 

1.00 

.64 

.45 

-.03 

.08 

.38 

.02 

-.37 

GRD 

.19 

.13 

.64 

1.00 

.82 

-.15 

.38 

.12 

.01 

-.05 

FLT 

-.01 

.50 

.45 

.82 

1.00 

-.39 

.58 

.12 

.02 

-.05 

CO 

.35 

-.54 

-.03 

-.15 

-.39 

1.00 

-.20 

.31 

-.07 

-.05 

VID 

.14 

-.002 

.08 

.38 

.58 

-.20 

1.00 

.58 

-.41 

-.53 

VIS 

.03 

-.44 

.38 

.12 

.12 

.31 

.58 

1.00 

-.52 

-.91 

STR 

-.11 

.51 

.02 

.01 

.02 

-.07 

-.41 

-.52 

1.00 

.56 

MO 

-.09 

.39 

-.37 

-.05 

-.05 

-.05 

-.53 

-.91 

.56 

1.00 

Note:  WT=total  error-point  score,  AGE=participant*s  age,  SEM=semesters  in  program,  GRD=number  of 
courses  taken,  FLT^number  of  flight  hours,  CO=computer  experience,  VID=video  game  experience, 
VIS=vision,  STR=stress,  MO=motivation 


ANOVA’s  were  also  conducted  on  the  post-session  questionnaire  variables  (workload,  self- 
assessment,  busyness,  stress,  thinking  and  concentration,  and  fatigue).  Means  for  post¬ 
session  questionnaire  variables  are  presented  in  table  4-7.  Reports  of  perceived  workload 
and  busyness  were  both  close  to  being  significantly  different  between  the  group  that  made 
fewer  errors  and  the  group  that  made  more  (workload:  F(l,10)=4.17,  p<.l  ;  busyness: 
F(l,10)=3.32,  p<.l). 


TABLE  4-7.  MEAN  RATING  AND  STANDARD  DEVIATION  FOR  EACH 
POST-SESSION  QUESTIONNAIRE  VARIABLE  FOR  EACH 
POST-HOC  TEST  GROUP  IN  EXPERIMENT  2 


Fewer  (<6)  Errors 

More  (>9)  Errors  f 

Mean 

SD 

Mean 

SD 

Workload 

9.00 

1.97 

11.00 

1.37 

Self-Assessment 

5.86 

2.27 

5.30 

1.67 

Busyness 

9.20 

1.21 

9.90 

.45 

Stress 

8.29 

1.70 

9.30 

.99 

Thinking  and  Concentration 

8.71 

1.65 

9.80 

.45 

Fatigue 

4.50 

2.45 

5.90 

2.88  1 

31 


Additionally,  a  multiple  regression  analysis  was  conducted  on  the  post-session  questionnaire 
variables  to  determine  whether  any  of  them  correlated  with  performance  independently  of 
other  variables  and  regardless  of  Ae  participant’s  group.  The  correlation  matrix  is  presented 
in  table  4-8.  This  analysis  found  that  worldoad  was  the  only  variable  that  correlated 
somewhat  significantly  with  error-point  total  (+.59,  p<.l).  Perceived  workload  tended  to 
increase  as  error-point  total  increased. 


TABLE  4-8.  CORRELATION  MATRIX  FOR  ALL  POST-SESSION 
QUESTIONNAIRE  VARIABLES  AND  PERFORMANCE 


WT 

SELF 

THINK 

WORK 

BUSY 

STR 

WT 

1.00 

-.22 

.39 

.20 

.51 

.43 

.49 

SELF 

-.22 

1.00 

-.60 

1 

o 

-.29 

-.52 

-.35 

THINK 

.39 

-.60 

1.00 

.63 

.79 

.86 

.87 

TIRED 

.20 

-.50 

.63 

1.00 

.35 

.37 

.57 

WORK 

.51 

-.29 

.79 

.35 

1.00 

.80 

.90 

BUSY 

.43 

-.52 

.86 

.37 

.80 

1.00 

.76 

STR 

.49 

-.35 

.87 

.57 

.90 

.76 

1.00 

Note:  WT=total  error-point  score,  SELF=self  assessment  rating,  THlNK=thinking  and  concentration, 
HREDs  fatigue,  WORK = workload,  BUSY= busyness,  STR=stress 

Strategies  reported  by  these  two  groups  of  participants  on  the  post-session  questionnaire 
variables  were  also  examined  to  see  whether  this  would  provide  more  insight  into  the  reasons 
for  their  performance  differences.  Specific  strategies  were  not  widely  cited  by  participants  in 
either  group.  No  differences  were  discemable  between  the  groups  in  terms  of  Aese  reports. 
Four  of  the  seven  participants  in  the  group  that  made  fewer  errors  cited  strategies,  three  of 
five  in  the  group  that  made  more  errors  cited  them.  All  of  the  strategies  that  were  reported 
referred  to  holding  aircraft  at  fixes  or  on  the  ground  or  indicated  the  use  of  data  tags. 

Holding  aircraft  at  fixes  and  keeping  aircraft  from  departing  allow  participants  to  regulate  the 
flow  of  aircraft.  Data  tags  allow  the  destination  of  each  aircraft  to  be  placed  directly  under 
its  call  letters  on  the  scope.  Tagging  allows  this  information  to  be  readily  accessible  in  that 
participants  do  not  have  to  keep  referring  to  the  electronic  flight  strips  at  the  side  of  the 
screen.  Holding  and  using  data  tags  were  cited  either  individually  or  in  combination  in 
strategy  reports. 
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Most  of  the  comments  referred  not  to  strategies  but  to  difficulty  in  keeping  up  with  the 
scenarios.  Seven  of  the  participants  specifically  indicated  that  there  were  "too  many"  aircraft 
to  direct  comfortably  and  effectively.  Four  of  them  were  from  the  group  that  made  fewer 
errors,  three  were  from  the  group  that  made  more.  Seven  participants  (not  necessarily  the 
same  seven  as  above)  indicated  Aat  it  was  hard  to  take  over  the  traffic  from  a  partner 
because,  as  one  explained,  "the  traffic  was  not  worked  the  way  I  would  have  worl^  it." 
Another  described  that  he  "tried  to  work  in  the  same  form  the  other  guy  was  doing."  Four 
of  these  seven  participants  were  from  the  group  that  made  fewer  errors,  three  were  from  the 
group  that  made  more  errors. 

In  summary,  participants’  reports  indicated  that  directing  traffic  in  this  experiment  was 
complicated,  either  because  of  the  number  of  aircraft,  or  for  some,  because  they  tried  to 
work  from  a  partner’s  strategy  or  set  up.  As  one  participant  responded,  his  strategy  was 
"No  strategy-out  of  control." 

4.6  DISCUSSION. 

The  purpose  of  Experiment  2  was  to  determine  whether  an  increase  in  available  planning 
time  prior  to  taking  control  of  a  scenario  improved  performance.  The  data  from  this 
experiment  do  not  support  this.  Differences  in  performance  between  sessions  for  which 
participants  had  had  S  minutes  of  time  to  view  an  on-going  scenario  were  not  significantly 
different  from  sessions  for  which  participants  had  had  just  2  minutes  of  viewing  time. 

As  indicated,  one  possible  reason  why  the  5-MINUTE  viewing  condition  did  not  result  in 
improved  performance  over  the  2-MINUTE  condition  may  lie  in  the  fact  that  the  5-MINUTE 
viewing  condition  was  always  tested  at  a  later  point  in  the  scenario.  Since  the  number  of 
aircraft  present  on  the  scope  was  affected  not  only  by  time  into  the  scenario,  but  also  by  the 
performance  of  the  person  relinquishing  control,  scenarios  could  potentially  become  very 
complicated  as  time  progressed.  For  example,  if  the  participant  initially  controlling  the 
scenario  made  contact  with  each  aircraft  immediately  and/or  had  difficulty  landing  or  handing 
off  aircraft,  then  the  number  of  aircraft  on  the  scope  may  have  exceeded  a  manageable  level 
for  those  who  then  took  over  control.  One  participant  indicated  that  there  were  16  aircraft 
on  the  scope  when  he  assumed  control.  This  volume  of  traffic  may  have  made  effective 
control  of  the  scenario  extremely  difficult.  This  difficulty  would  be  expected  to  be  especially 
apparent  for  novices.  The  2-MINUTE  viewing  condition,  on  the  other  hand,  was  always 
tested  early  in  the  scenario  when  traffic  was  potentially  much  lighter.  During  the  first  7 
minutes  of  the  scenario,  eight  aircraft  had  been  programmed  to  enter  the  airspace;  five 
arrivals,  two  departures,  and  one  overflight.  Even  if  the  participants  who  initially  controlled 
this  scenario  made  contact  with  each  aircraft  immediately  and  were  unable  to  hand-off  or 
land  any  of  them  during  this  time  period,  the  maximum  number  that  could  have  been  present 
on  the  scope  when  their  partners  took  control  would  have  been  eight.  While  the  complexity 
of  a  traffic  situation  can  be  high  even  for  this  number  of  aircraft,  it  would  most  likely  be 
lower  than  the  complexity  of  a  pattern  emerging  from  a  combination  of  16  aircraft.  An 
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increase  in  planning  time  may  have  been  useful,  but  its  effects  may  have  been  masked  by  the 
greater  complexity  of  the  traffic  situation  during  which  it  was  test^. 

The  overall  difficulty  of  the  scenarios  used  in  Experiment  2  was  reflected  in  participants’ 
post-session  questionnaire  responses.  Ratings  for  workload,  busyness,  thinking  and 
concentration,  and  stress  were  very  high.  The  mean  rating  was  close  to  the  level 
response  for  each.  This  ceiling  effect  made  it  impossible  to  discern  a  difference  in 
participant’s  impressions  of  the  two  viewing  conditions  since  both  were  rated  as  requiring 
close  to  maximum  effort.  Also  supporting  this,  was  the  result  that  all  of  the  variables  from 
the  post-session  questionnaire  of  Experiment  2  differed  from  those  of  Experiment  1. 

Perceived  level  of  workload,  busyness,  thinking  and  concentration,  stress,  and  fatigue  were 
reported  higher  for  Experiment  2.  Self-assessment  of  performance  was  rated  lower. 

The  apparent  difficulty  that  participants  had  in  Experiment  2  was  also  observed  by  the  ATCS 
experimenter  as  well.  He  noted  "confusion”  and  "lack  of  an  organized  plan  and  control 
strategy"  on  the  part  of  many  students.  It  was  observed,  for  example,  that  aircraft  were 
sometimes  aimed  directly  at  one  another.  A  number  of  stress  indicators  were  also  apparent: 
fidgeting,  sweating,  nail-biting,  and  moving  closer  to  the  screen.  One  student  was  observed 
to  "freeze." 

The  scenarios  used  in  Experiment  2  may  have  been  too  complex  for  participants  to  develop  a 
strategy.  Some  comments  made  by  participants  for  example,  indicate  that  they  found  it 
difficult  to  take  over  control  from  a  partner  because  they  were  unable  to  develop  and 
implement  their  own  plans  quickly  enough.  This  view  is  supported  by  Whitfield’s  (1979) 
work  in  which  he  assessed  that  "Ae  relief  must  ensure  establishing  his  own  picture"  0[>.  21) 
in  order  for  control  to  be  successful.  He  further  discussed  that  problems  should  be  dealt 
with  by  the  controller  leaving  the  position  before  the  next  controller  takes  over  in  order  to 
promote  effective  transition.  This  was  not  necessarily  the  situation  in  the  current  experiment. 
In  this  experiment,  participants  were  required  to  take  over  at  a  particular  time,  regardless  of 
whether  their  partner  was  in  the  midst  of  dealing  with  one  or  several  problems.  While  this 
situation  may  not  be  likely  to  occur  in  an  actual  ATC  situation,  it  was  used  in  an  attempt  to 
directly  investigate  the  effect  of  planning  time. 

An  ANOVA  conducted  on  the  preliminary  questionnaire  variables  did  not  indicate  that  the 
participants  in  the  experimental  groups  differed  significantly  from  one  another  on  most  of 
these  variables.  The  only  variable  for  which  the  difference  was  somewhat  significant  was 
stress.  Those  who  performed  under  the  2-MlNUTE  condition  first  had  reported  a  somewhat 
lower  level  of  stress  coming  into  the  experiment  than  those  who  performed  under  the  S- 
MINUTE  condition  first. 

For  the  post-hoc  analyses,  participants  were  divided  into  groups  based  on  their  overall 
performance  (error  totals  less  than  6  and  greater  than  9)  to  investigate  whether  performance 
could  be  attributed  to  factors  that  were  not  measured  in  the  test  conditions.  An  ANOVA 
revealed  a  significant  difference  for  the  interaction  of  group  by  condition.  The  group  with 
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fewer  overall  errors,  tended  to  reduce  errors  between  the  2-MINUTE  and  5-MINUTE 
conditions,  while  those  with  a  greater  number  of  overall  errors  made  more  errors  under  the 
5-MINUTE  than  the  2-MINUTE  conditions.  This  suggests  that  those  who  were  more 
effective  in  managing  traffic  overall  were  less  affected  by  the  increased  complexity  of  the 
traffic  situation  arising  in  the  5-MINUTE  condition  than  those  who  were  less  effective  in 
managing  traffic. 

The  preliminary  and  post-session  questionnaire  variables,  as  well  as  reported  strategies,  were 
analyzed  to  investigate  whether  they  revealed  any  differences  between  these  groups.  No 
differences  were  found  between  them  for  the  preliminary  questionnaire  variables.  These 
were  investigated  because  it  was  reasoned  that  some  of  the  differences  in  their  performance 
may  have  been  attributed  to  differences  in  flight  experience  or  in  the  number  of  aviation 
courses  or  semesters  completed.  It  seemed  likely  that  those  making  fewer  errors  may  be 
those  indicating  a  greater  level  of  experience  on  these  variables.  However,  this  was  not 
observed.  A  comparison  of  post-session  questionnaire  variables  indicated  that  these  groups 
differed  somewhat  in  their  reports  of  workload  and  busyness.  The  group  that  made  more 
errors  overall  reported  somewhat  higher  average  levels  of  workload  and  busyness  for  the 
sessions  than  did  the  group  making  fewer  errors.  The  subjective  impressions  for  this 
measure,  therefore,  corresponded  with  objective  performance.  Additionally,  multiple 
regression  analyses  were  conducted  to  determine  the  relationship  between  Aese  variables  and 
individual  performance,  regardless  of  group.  None  of  the  partial  correlations  of  preliminary 
questionnaire  variables  with  performance  were  significant  at  either  p<  .05  or  p<  .1.  A 
regression  analysis  conducted  on  the  post-session  questionnaire  variables  and  performance 
indicated  that  only  the  partial  correlation  for  perceived  workload  and  performance  was 
somewhat  significant  (p<  .1).  This  suggests  that  those  who  made  more  errors  found  the  task 
to  be  more  demanding. 

One  explanation  for  the  differences  observed  in  performance  may  again  have  been  due  to  the 
use  of  different  individual  strategies.  As  in  Experiment  1,  specific  strategies  were  not  taught 
to  participants.  The  intent  of  the  experiment  was  only  to  examine  the  effect  of  increased  and 
decreased  planning  time  availability  on  performance.  Participants  were  free  to  develop  their 
own  strategies,  some  of  which  may  have  been  more  effective  than  others.  To  more  carefully 
examine  speciHc,  individual  strategies  it  would  have  been  necessary  to  monitor  each 
participant’s  performance  during  a  session  and  to  ask  him/her  to  verbally  describe  thoughts 
and  actions  as  he/she  proceeded.  This  was  not  possible  given  the  group  test  environment  in 
which  this  experiment  was  conducted.  Strategies  were  asked  about  in  the  post-session 
questionnaire  and  these  were  examined  to  try  to  get  a  sense  of  the  individual  strategies  used. 
However,  most  participants  focused  on  the  difficulties  they  had  in  working  with  the  more 
difficult  scenarios  used  in  this  experiment  rather  than  on  the  kinds  of  plans  and  activities  that 
they  engaged  in.  The  most  common  strategy-oriented  comments  dealt  with  using  "holds"  or 
"data  tags."  Holds  kept  aircraft  from  requiring  immediate  action  and  could,  therefore,  be 
used  to  keep  the  number  of  aircraft  in  ne^  of  control  to  a  more  manageable  level.  It  was 
one  direct  way  in  which  participants  could  regulate  the  flow  of  traffic  to  a  level  that  each 
found  person^ly  easier  to  handle.  Consequently,  the  amount  of  information  to  be  maintained 


35 


in  working  memory  was  reduced.  Participants  using  this  strategy  indicated  that  they  realized 
that  their  memory  capacity  for  more  aircraft  was  reaching  its  limit  and  took  direct  action  to 
bring  the  number  that  they  had  to  work  with  under  control.  Data  tags  allowed  participants  to 
include  a  meaningful  piece  of  information  about  an  .  aircraft  right  on  the  screen  at  the  location 
of  the  aircraft  call  letters.  Tags  were  usually  given  to  indicate  destination  (i.e.,  the  airport 
the  aircraft  was  to  land  at  or  the  fix  at  which  it  needed  to  be  handed  off).  These  data  tags 
reduce  the  need  to  have  to  take  one’s  eyes  away  from  the  screen  to  search  for  the  needed 
information  from  the  flight  strips.  Both  appear  useful  given  the  apparent  demands  of  this 
experimental  situation. 

There  were  no  observable  differences  between  the  groups  in  terms  of  the  proportion  of 
participants  in  each  group  who  cited  using  holds  or  data  tags.  If  data  tags  and  holds  were 
critical  to  performance,  a  greater  proportion  of  participants  in  the  group  making  fewer  errors 
may  have  been  expected  to  indicate  using  them  than  the  group  making  more  errors.  The  fact 
that  this  result  was  not  obtained  does  not  necessarily  negate  their  effect.  These  strategies 
may,  for  example,  have  been  used  as  part  of  broader  strategies,  some  of  which  were  more 
effective  than  others.  Participants  did  not  elaborate  on  this,  nor  did  they  report,  for 
example,  how  often  these  were  used.  Some  may  have  used  them  frequently  and  others  less 
frequently.  Frequency  of  use  is  another  variable  that  may  have  been  important  in 
distinguishing  the  groups  but  it  was  not  measured  in  this  experiment. 

Beyond  identifying  the  use  of  holds  or  data  tags,  specific  strategies  were  not  reported  by 
participants.  It  was,  therefore,  not  possible  to  relate  specific  strategies  to  performance. 

In  summary,  this  experiment  did  not  find  evidence  that  the  amount  of  planning  time  made  a 
difference  in  performance.  Participant’s  performance  varied  widely  regardless  of  condition. 
In  general,  participants  indicated  that  both  test  conditions  were  very  difficult.  Responses  on 
post-session  questionnaire  variables  for  both  conditions  were  at  nearly  the  highest  level 
allowed  for  workload,  busyness,  thinking  and  concentration,  and  stress.  Observations  of 
participant’s  performance  suggested  difficulty  as  well  in  that  participants  found  that 
Experiment  2  involved  a  higher  level  of  workload,  stress,  and  thinldng  and  concentration 
than  Experiment  1  and  also  resulted  in  more  stress,  greater  fatigue,  and  a  lower  level  of  self- 
assessment. 

Participants  were  also  grouped  according  to  the  total  number  of  errors  made  to  determine 
whether  other  variables  could  help  account  for  differences  in  performance.  The  variables 
measured  and  analyzed  in  this  experiment  did  not  reveal  which  factors  may  have  been  most 
important  in  determining  performance  level.  Individual  strategies,  which  could  not  be 
assessed  given  the  design  of  this  experiment,  may  provide  the  underlying  critical  information 
about  performance  differences.  Future  experiments  are  needed  which  test  participants  in  a 
manner  that  better  enables  investigators  to  determine  what  these  strategies  are. 
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5.  EXPERIMENTS:  EFFECT  OF  PHYSICAL  ACTIVITY  (WRITING)  ON  MEMORY 
FOR  ISSUED  COMMANDS. 


5.1  PURPOSE. 

The  purpose  of  the  third  experiment  was  to  investigate  the  effect  of  note-writing  on  the 
controller’s  ability  to  remember  commands  issued  to  aircraft.  Prior  reports  have  suggested 
that  note-writing  is  vital  for  maintaining  memory  of  critical  information  which,  in  turn, 
contributes  to  "keeping  the  picture." 

5.2  METHOD. 

5.2.1  Participants. 

Thirteen  students  participated  in  Experiment  3,  all  had  served  in  at  least  one  session  of  one 
of  the  other  two  experiments.  This  meant  that  the  data  from  the  preliminary  questionnaire 
were  quite  similar  to  those  obtained  in  Experiments  1  and  2.  The  ages  of  the  participants 
ranged  from  18  to  26  (mean =20.46,  SD=2.47).  They  had  completed  one  to  four  semesters 
in  the  aviation  program  (mean =2.38,  SD=1.19)  and  from  one  to  eight  basic  ground  courses 
(mean=3.46,  SD=2.73).  Number  of  flight  hours  completed  ranged  from  0  to  250 
(mean =108.54,  SD =93.23).  Three  participants  indicated  having  had  some  aviation 
experience  prior  to  attending  Mercer  County  Community  College  but  did  not  elaborate  on 
what  that  involved. 

Participant’s  ratings  (1  =lowest,  10=highest)  on  their  level  of  computer  experience  ranged 
from  3  to  9  (mean=5.46,  SD=2.03),  video  game  experience  rang^  from  5  to  10 
(mean =7.77,  SD=1.79),  and  quality  of  vision  ranged  from  5  to  10  (mean=9.31, 

SD=1.44).  None  indicated  any  color  vision  deficiency.  Participants  indicated  high 
agreement  in  their  willingness  to  participant  in  the  study  (mean=9.85,  SD=.55).  They  also 
indicated  high  ratings  for  their  general  health  (mean =9.54,  SD=.88).  They  did  indicate 
variable  levels  of  recent  stress  (mean =4.69,  SD=2.59).  Level  of  motivation  to  participate 
was  generally  high  (mean =1.46,  SD=1.39).  This  question  was  worded  so  that  1  was  equal 
to  the  highest  level  of  agreement,  10  the  lowest. 

5.2.2  Test  Conditions. 

Two  experimental  conditions  were  to  be  tested  to  evaluate  the  effect  of  note-writing  on 
memory  for  issued  commands.  However,  due  to  a  misunderstanding  of  the  instructions, 
several  participants  did  not  carry  out  the  NO- WRITING  condition  as  intended.  The  NO- 
WRITING  condition  was  eliminated  as  described  below. 

5.2.2. 1  WRITING  Condition. 

Under  this  condition,  participants  worked  in  pairs  with  half  of  the  participants  tested  at  a 
time.  Participants  were  instructed  that  they  were  to  write  down  each  command  as  it  was 
issued  on  the  paper  flight  strips  provided  for  16  aircraft  that  were  to  enter  the  airspace  over  a 
27  minute  test  session  (see  appendix  C,  pages  C-1  and  C-2).  Actually,  test  sessions  were 
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only  conducted  for  13  minutes,  but  more  flight  strips  were  included  so  that  it  would  be 
difficult  for  participants  to  anticipate  which  aircraft  would  mark  the  end  of  the  session.  A 
total  of  1 1  aircraft  were  actually  tested  in  Experiment  3.  The  note-writing  activity  was 
intended  to  simulate  the  way  that  paper  flight  strips  are  often  used  by  actual  air  traffic 
controllers.  Farmers  also  recorded  actions  independently  to  monitor  accuracy.  Sheets  on 
which  parmers  indicated  actions  are  shown  in  appendix  C,  pages  C-3  and  €-4.  As  always, 
participants  were  instructed  to  work  towards  minimizing  errors  during  each  session. 

After  13  minutes  of  testing,  participants  were  instructed  to  halt  the  scenario  and  to  indicate 
on  a  response  sheet  all  of  the  actions  performed  for  each  of  the  aircraft  listed.  Response 
sheets  are  shown  in  appendix  C,  pages  C-S  and  C-6. 


Participant’s  performance  was  measured  by  comparing  each  response  to  the  information 
provided  by  parmers  so  that  the  percentage  of  correct  responses  could  be  calculated.  A 
measure  of  flight  strip  use  was  also  obtained  by  comparing  the  commands  written  on  the 
strips  by  participants  to  those  recorded  by  partners. 


52.2.2 


:NG  condition. 


Under  this  condition,  participants  also  worked  in  pairs,  with  half  of  the  participants  tested  at 
a  time.  Only  computer-displayed  flight  strips  were  available  to  participants.  They  were  not 
provided  with  paper  flight  strips  and  were  not  allowed  to  write  their  commands.  Farmers 
recorded  these  actions  independently  for  scoring  purposes.  Farticipants  were  told,  as  always, 
to  minimize  errors  during  the  session. 

As  in  the  WRITING  condition,  participants  were  required  to  halt  the  scenario  and  report  the 
commands  issued  for  each  aircraft  after  13  minutes  of  testing.  Farmers’  records  and 
participants’  response  sheets  were  collected  following  the  session  so  that  the  pmeentage  of 
correct  responses  could  be  calculated. 

5.3  DESIGN. 


Farticipants  were  to  be  tested  according  to  a  repeated  measures  design  in  which  the 
WRITING  and  NO-WRITING  conditions  serv^  as  the  two  levels  of  the  tested  factor. 
Unlike  previous  experiments.  Experiment  3  used  percentage  of  correct  responses  for 
memories  of  specific  actions  as  the  primary  dependent  variable,  rather  than  an  error-point 
total. 


Experimental  testing  was  conducted  on  2  separate  days.  On  the  first  day,  all  of  the 
participants  were  tested  under  the  NO-WRITING  condition.  One-half  of  the  participants 
worked  first,  while  the  other  half  acted  as  parmers.  For  the  second  half  of  the  session, 
participants  performed  the  alternate  role.  On  the  second  day  of  testing  all  participants  were 
tested  under  the  WRITING  condition.  Again,  one-half  of  the  participants  worked  first,  while 
the  other  half  acted  as  partners,  and  for  the  second  half  of  the  session,  participants  performed 
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the  alternate  role.  Participants  completed  the  post-session  questionnaire  following  each 
session. 

Some  of  the  participants  were  unclear  about  the  instructions  for  the  NO-WRITING  condition. 
They  wrote  out  actions  while  working  during  the  session.  This  happened  because  score 
sheets  were  distributed  to  participants  prior  to  the  start  of  the  session  while  the  instructions 
were  being  given.  These  instructions  informed  them  that  they  would  be  asked  to  write  out 
each  of  the  commands  they  issued  to  aircraft  after  the  experimenter  indicated  that  the  session 
had  ended.  However,  a  number  of  them  began  writing  on  the  available  sheets  as  they  were 
working.  Since  it  could  not  be  accurately  determined  how  much  information  was  written 
during  the  session,  the  entire  session  was  subsequently  eliminated.  This  made  it  necessary  to 
investigate  the  effect  of  note- writing  on  memory  differently  than  originally  intended.  To 
examine  this  effect,  correlations  between  the  proportion  of  notetaking  and  proportion  of 
remembered  commands  were  conducted  for  data  in  the  WRITING  condition  alone.  During 
the  WRITING  session,  response  sheets  were  distributed  after  the  session  so  that  participants 
would  not  be  able  to  mark  anything  on  them  until  that  time.  The  flight  strips  on  which  they 
did  record  their  actions  during  the  session  were  taken  away  from  them  immediately  after  the 
session  so  that  they  would  not  have  access  to  their  "notes." 

Two  programmed  scenarios,  identical  except  for  the  call  letters  of  the  aircraft,  were  used 
during  the  test  session.  They  were  created  as  described  in  the  Equipment  section  above. 
Scenarios  were  programmed  so  that  16  aircraft  would  enter  the  airspace  in  27  minutes. 
However,  only  the  first  11  aircraft  were  actually  included  in  the  13  minute  test  sessions,  9 
arrivals,  2  departures. 

As  in  previous  experiments,  information  from  the  post-session  and  preliminary  questionnaires 
was  also  analyzed  to  determine  the  extent  to  which  these  variables  were  related  to 
performance. 

5.4  RESULTS. 

Partners  wrote  out  each  of  the  actions  taken  by  those  controlling  the  scenario.  This  provided 
a  record  of  the  actual  commands  issued.  The  percentage  of  remembered  commands  was 
calculated  by  determining  the  total  number  of  correct  responses  indicated  on  response  sheets 
divided  by  the  total  number  of  actions  recorded  by  partners.  Overall,  the  percentage  of 
remembered  commands  ranged  from  0  to  .81  (mean =.39,  SD=.30). 

Flight  strip  use  was  calculated  to  determine  whether  note  writing  correlated  with  remembered 
commands.  Flight  strip  use  was  determined  by  totalling  the  number  of  actions  indicated  by 
participants  on  the  flight  strips  and  dividing  by  the  number  of  actions  recorded  by  partners. 
Flight  strip  use  ranged  from  .23  to  1.0  (mean=.67,  SD=.28).  The  percentage  of 
remembered  commands  was  positively  and  significantly  correlated  with  flight  strip  use,  r= 
-I-.82,  p<  .(X)l.  These  data  are  plotted  in  figure  5-1.  This  result  suggests  that  those  who 
tended  to  write  more  of  the  issued  commands  on  flight  strips  while  working  also  tended 
to  remember  more  of  those  commands  after  the  session  was  completed. 
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%  Flight  Strip  Use 

HGURE  5-1.  PLOT  OF  PERCENTAGE  OF  REMEMBERED  COMMANDS  WITH 
PERCENTAGE  OF  FLIGHT  STRIP  USE 

During  this  experiment,  participants  were  not  asked  to  monitor  their  individual  errors  while 
working  as  they  had  during  Experiments  1  and  2.  Since  this  experiment  was  concerned  with 
the  effects  of  note  writing  on  flight  strips,  only  note  writing  was  required  during  the  session. 
It  would  not  have  been  feasible  for  participants  to  write  notes  and  indicate  errors  during  the 
session  since  this  would  have  taken  too  much  time  away  from  actually  controlling  traffic. 
TRACON  II’s  post-session  performance  evaluation  was  used  to  obtain  information  on  the 
number  of  crashes,  number  of  missed  approaches,  and  number  of  handoff  errors  as  well  as 
the  total  performance  score  and  the  number  of  aircraft  worked  during  the  session.  Since 
participants  made  relatively  few  errors  during  this  session  (6  of  13  participants  made  no 
errors),  these  variables  would  have  been  inadequate  as  dependent  measures.  Instead, 
TRACON  performance  scores  and  the  total  number  of  aircraft  worked  were  used  as  measures 
of  performance. 

TRACON  performance  scores  are  based  on  control  efficiency.  Points  are  deducted  wh«i 
aircraft  are  diverted  from  their  most  direct  route  and  added  when  aircraft  are  landed  or 
handed  off  successfully.  Point  values  are  weighted  to  reflect  the  severity  of  errors  (i.e., 
more  points  are  deducted  for  separation  conflicts  than  for  handoff  errors).  The  maximum 
score  possible  varies  as  a  function  of  the  number  of  aircraft  programmed  into  a  scenario  per 
unit  time.  Including  more  aircraft  per  unit  time  allows  for  a  greater  potential  score,  but  also 
makes  the  scenario  more  complicated,  increasing  the  chance  for  errors.  Final  performance 
scores  can  be  either  positive  or  negative.  In  Experiment  3,  performance  scores  ranged  from 
200  to  4760.  To  simplify  the  analyses,  scores  were  divided  by  1000  and  rounded  to  the 
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nearest  tenth.  Thus,  the  converted  scores  for  Experiment  3  ranged  from  .2  to  4.8  with  a 
mean  of  2.37  (SD= 1.79). 

The  number  of  aircraft  worked  refers  to  the  total  number  of  aircraft  that  the  participant  was 
able  to  successfully  direct  to  appropriate  destinations.  The  number  of  aircraft  worl^  in  this 
experiment  ranged  from  0  to  5  (mean=2.92,  SD=1.85).  A  low  number  was  not 
unexpected.  Since  the  test  session  involved  11  aircraft  entering  the  airspace  over  13  minutes, 
there  was  not  enough  time  for  most  of  these  aircraft  to  travel  completely  to  their  destinations. 

Performance  scores  were  positively  correlated  with  flight  strip  use  (r=+.73,  p<.01,  see 
figure  5-2).  Participants  who  wrote  on  strips  more  tended  to  have  better  performance 
scores  than  those  who  wrote  less.  One  participant  experienced  a  crash,  and  as  a  result  did 
not  receive  a  score.  His  data  are  not  included  in  the  analyses  but  are  reported  sqiarately 
below. 


nOXJRE  5-2.  PLOT  OF  PERFORMANCE  SCORE  WITH  PERCENTAGE  OF 
FLIGHT  STRIP  USE 
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The  percentage  of  remembered  commands  was  also  strongly  correlated  with 
performance  score  (r=  +  .95,  p  <  .001).  The  better  the  participant’s  score,  the  more 
he/she  remembered  (see  figure  S-3). 


%  Remembered  Commands 


nCURE  5-3.  PLOT  OF  PERFORMANCE  SCORE  WITH  PERCENTAGE  OF 
REMEMBERED  COMMANDS 


The  percentage  of  remembered  commands  was  significantly  coirdated  with  the  number  of 
aircraft  worked  successfully  during  the  session  (+.89,  p<.001).  This  suggests  that  the 
higher  proportion  of  remembered  commands  did  not  result  because  these  participants  had  . 
fewer  commands  to  issue  (see  figure  S-4). 

Workload  assessments  were  negatively  and  significantly  correlated  with  flight  strip  use 
(r  =-.71,  p<.01).  Those  who  wrote  more  on  strips  during  the  session  indicated  having 
less  workload  than  those  who  used  them  less.  Two  other  post-session  questionnaire 
variables  were  also  correlated  with  performance.  Stress  was  negatively  correlated  with  the 
percentage  of  remembered  commands  (r=-.64,  p<.01).  Memory  for  more  commands  was 
related  to  a  lower  level  of  reported  stress.  Score  was  also  negatively  correlated  with  stress 
(r=  -.61,  p<  ,01).  The  higher  the  score,  the  lower  the  reported  level  of  stress.  Self- 
assessment  scores  were  positively  and  significantly  correlated  with  the  number  of  aircraft 
worked  (r=  +  .67,  p<  .01).  The  more  aircraft  successfully  handled,  the  better 
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self-assessment  reports  tended  to  be.  None  of  the  other  post-session  questionnaire  variables 
were  significantly  correlated  with  any  of  the  performance  measures  (flight  strip  use, 
percentage  of  remembered  commands,  performance  score  or  number  of  aircraft  worked, 
p>.l). 


Number  of  Aircraft  Worked 


nCURE  5-4.  PLOT  OF  PERCENTAGE  OF  REMEMBERED  COMMANDS  WITH 
NUMBER  OF  AIRCRAFT  WORKED 


Multiple  regression  analyses  indicated  that  none  of  the  variables  measured  by  the  preliminary 
questionnaire  were  significantly  correlated  with  any  of  the  performance  measures  (percentage 
of  remembered  commands,  performance  score,  flight  strip  use,  or  number  of  aircraft 
worked,  p>.l). 

As  noted  above,  one  participant  experienced  a  crash  and  as  a  result,  no  TRACON 
performance  information  was  available  for  him.  A  separate  examination  of  his  flight  strips 
and  response  sheet  indicated  that  he  used  flight  strips  minimally  (.27).  He  was  also  unable 
to  indicate  any  of  the  specific  actions  he  had  taken  for  the  aircraft  on  his  response  sheet.  He 
rated  workload  at  the  highest  level  possible  (12),  as  well  as  busyness  (10),  how  much 
thinking  and  concentration  he  needed  (10),  and  how  much  stress  he  experi«iced  during  the 
session  (10).  He  evaluated  his  performance  at  the  lowest  possible  levd  (1).  In  his 
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post-session  comments  he  did  not  cite  any  reasons  for  this  performance.  This  participant 
had,  however,  less  experience  than  most  others  in  that  he  had  only  completed  one-half  of 
Experiment  1  in  addition  to  the  training  sessions  prior  to  this  experiment. 

5.5  DISCUSSION. 

The  results  of  Experiment  3  suggest  that  an  increase  in  flight  strip  use  is  associated  with 
better  memory  for  critical  information.  Writing  notes  about  issu^  commands  on  the  strips 
while  working  through  the  session  correlated  with  better  recall  of  these  commands  after  the 
session  was  completed  when  all  reference  information  had  been  removed  from  the  scope. 
Performance  scores  also  tended  to  increase  with  flight  strip  use  and  with  memory  for 
commands. 

The  results  indicated  that  better  memory  for  commands  was  not  likely  to  have  been  due  to 
having  fewer  total  commands  to  remember  since  memory  for  commands  was  also  correlated 
with  the  number  of  aircraft  worked.  Having  a  greater  number  of  aircraft  to  control  would 
necessarily  require  more  commands. 

The  results  also  indicated  that  workload  evaluations  were  lower  for  participants  who  used 
flight  strips  more.  The  perception  of  having  expended  a  lower  level  of  effort  while  working 
through  a  session  was  associated  with  flight  strip  use.  Flight  strip  use  may,  perhaps,  have 
promoted  better  organization  of  critical  information  for  each  aircraft,  allowing  the 
participants  who  used  them  more  effective  control.  Flight  strips  have  been  suggested  as 
useful  for  this  reason  by  others  (Hopkin,  1992). 

The  results  of  this  final  study  are  promising  in  that  they  indicate  a  relationship  between  flight 
strip  use  and  both  memory  and  performance.  However,  correlations  alone  do  not  provide 
sufficient  support  for  this  since  they  do  not  describe  cause-effect  relationships.  It  remains  to 
be  determined  whether  writing  on  flight  strips  was  responsible  for  the  increase  in  recall.  It 
cannot  be  ruled  out,  for  example,  that  "better"  participants  are  those  who  will  not  only  score 
higher,  but  who  also  have  time  to  devote  to  note  writing  and/or  who  have  time  to  organize 
and  store  information  more  effectively  for  later  recall. 

By  the  same  token,  "poorer"  participants  may  become  easily  overwhelmed,  and  have  less 
attention  to  devote  to  note  writing.  One  interpretation  of  the  subjective  workload 
assessments  suggests  this.  "Better"  participants  tended  to  report  a  lower  level  of  perceived 
workload  while  those  who  performed  more  poorly  tended  to  report  a  higher  level  of 
workload.  The  relationship  between  perceived  workload  and  flight  strip  use  has  also  been 
noted  for  actual  air  traffic  controllers.  In  Gromelski,  Davidson,  and  Stein’s  (1992)  report, 
half  of  the  170  controllers  interviewed  indicated  that  their  use  of  flight  strips  either  stopped 
or  was  substantially  reduced  under  high  workload  conditions.  Under  high  levels  of  perceived 
workload,  controllers  tend  to  look  for  shortcuts.  For  example,  to  reduce  memory  demands 
they  may  refuse  aircraft  requests  or  move  traffic  along  rigid  paths,  thereby  reducing  the 
number  of  actions  needed  per  aircraft. 
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Yet,  despite  the  fact  that  flight  strip  use  is  reported  to  decrease  with  perceived  workload, 
strip  marking  has  still  been  cited  by  controllers  as  the  most  frequently  used  memory  aid, 
even  under  the  busiest  conditions  (Gromelski  et  al.,  1992).  While  writing  increases  physical 
workload,  it  may  actually  help  to  keep  mental  workload  manageable.  To  this  end,  note 
writing  was  suggested  as  a  technique  that  controllers  could  use  to  keep  memory  lapses  from 
occurring  by  The  Controller  Memory  Handbook  (Stein  and  Bailey,  1989).  This  guide  to 
memory  suggested,  "Even  if  you  are  busy,  write  it  down",  offering  controllers  a  method  to 
help  reduce  the  mental  effort  required  to  keep  all  essential  components  of  a  current  air  traffic 
situation  readily  available  without  having  to  push  memory  to  its  limits. 

6.  CONCLUSIONS. 

The  three  experiments  conducted  during  this  stage  of  testing  served  as  initial  investigations 
into  the  role  of  two  types  of  memory  strategies  (planning  strategies  and  flight  strip 
management)  that  have  emerged  as  potentially  the  most  useful  in  the  air  traffic  control  (ATC) 
environment. 

Novice  participants  with  some  aviation  experience  were  tested  in  all  experiments.  These 
experiments  were  designed  so  that  many  participants  could  be  tested  simultaneously  in  a 
limited  span  of  time.  Participants  were  required  to  complete  several  hours  of  training  in 
order  to  understand  enough  about  how  to  control  traffic  before  they  were  able  to  participate 
in  the  experiments.  Thirteen  students  completed  training  and  participated  in  the  test  sessions. 
This  was  less  than  the  number  that  was  expected  to  be  necessary  to  obtain  statistically 
significant  results.  In  addition,  participant’s  performance  levels  also  varied  considerably 
from  one  to  another.  These  two  factors  made  it  difficult  to  be  able  to  discern  how  the 
experimental  conditions  may  have  affected  performance. 

The  variability  of  participants’  performance  was  initially  observed  during  the  training 
sessions  and  carried  through  to  the  test  sessions.  A  few  participants  seemed  to  be 
consistently  more  efficient  than  others.  One  of  the  experimenters,  an  air  traffic  control 
specialist  (ATCS),  noted  some  of  the  qualities  that  he  observed  in  these  participants.  To 
summarize,  the  most  efficient  performers: 

a.  Used  data  tags  for  airport  identification 

b.  Used  direct  clearances  to  destinations 

c.  Minimized  keystroke  entries 

d.  Memorized  essential  information 

e.  Used  vertical  separation  in  lieu  of  assuming  separation  would  exist 

f.  Immediately  took  action  to  send  arriving  aircraft  towards  destinations 

g.  Were  able  to  use  strips  as  a  notepad 

h.  Used  strips  to  forecast  upcoming  traffic 

i.  Used  strips  to  assist  in  visualizing  flight  plan  routes  and  as  an  aid  in  preplanning 

control  actions 

j.  Used  pending  information  as  an  indicator  of  projected  traffic  volume 
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k.  Effectively  utilized  time:  Prioritized 

l.  Displayed  as  much  information  as  needed  on  the  scope  to  minimize  the  need  to  refer 

to  more  distantly  located  flight  strips 

m.  Effectively  moved  data  tags  to  eliminate  data  block  overlap 

n.  Used  standardized  routings  to  destinations 

Experiment  1  was  designed  to  determine  whether  the  opportunity  to  plan  enhanced 
performance.  This  was  intended  to  serve  as  an  initial  investigation  into  the  general  role  of 
planning  in  ATCS  performance.  The  results  of  the  present  experiment  did  not  indicate  that 
having  the  opportunity  to  plan  was  more  beneficial  than  not  having  the  opportunity. 

However,  ratings  concerning  the  level  of  thinking  and  concentration  needed  to  work  through 
these  sessions  indicated  that  participants  did  find  that  the  NO-PLANNING  condition  was 
more  demanding.  Several  participants  acknowledged  a  preference  for  using  paper  flight 
strips. 

Possible  reasons  for  the  lack  of  significance  between  performance  under  the  two  test 
conditions  were  cited.  One  was  that  the  scenarios  used  in  the  experiment  appeared  to  have 
been  relatively  easy  for  the  majority  of  participants  to  work  with.  A  much  lower  number  of 
errors  were  made  overall,  compared  to  what  was  expected.  For  most  participants  very  few 
errors  were  made  under  either  condition,  making  it  difficult  to  identify  a  difference  between 
conditions.  A  second  may  have  been  due  to  the  variability  in  participants  performance,  since 
a  few  participants  did  experience  much  more  difficulty  than  others.  Other  factors  may, 
therefore,  have  affected  performance.  When  participants  were  separated  into  two  groups 
based  on  their  performance  levels,  the  results  indicated  that  those  who  performed  better 
entered  the  experiment  with  a  lower  level  of  reported  stress  and  a  higher  level  of  rqiorted 
aviation  course  experience  than  those  who  performed  more  poorly.  These  factors  may  have 
influenced  participants’  control  abilities  more  directly  or  more  strongly  than  did  the  different 
test  conditions. 

The  variability  of  participants’  performance  also  suggests  that  individual  strategies  may  have 
played  an  important  role  in  determining  performance.  Since  specific  strategies  were  not 
taught,  participants  were  free  to  develop  them  on  their  own.  Given  the  group  testing 
methods  used  in  the  present  experiments,  it  was  not  possible  to  determine  what  strategies 
individuals  were  using.  Some  may  have  been  able  to  plan  and  implement  more  effective 
strategies  in  comparison  to  others.  Others  may  simply  have  been  unable  to  develop  strategies 
at  all,  perhaps  because  they  felt  their  knowledge  in  this  area  was  too  limited. 

Participants  were  asked  to  indicate  what  strategies  they  had  used  on  the  post-session 
questionnaire.  However,  most  participants  were  not  very  specific  in  their  reports.  A  lack  of 
experience  in  providing  an  introspective  analysis  of  their  mental  operations  may  have  made  it 
difficult  for  them  to  have  been  more  elaborate.  Hie  technique  of  assessing  one’s  cognitive 
activities  typically  requires  some  practice,  especially  for  those  who  are  not  accustomed  to 
providing  information  of  this  kind. 
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Experiment  2  was  designed  to  determine  whether  increased  planning  time  enhanced 
performance.  Again,  the  results  of  this  experiment  were  not  statistically  significant.  The 
results  indicated  that  having  more  planning  time  did  not  reduce  errors  relative  to  a  condition 
in  which  less  planning  time  was  allowed.  The  scenarios  developed  for  Experiment  2  were 
programmed  to  include  more  aircraft  and  more  conflicts  than  those  used  in  Experiment  1. 
This  was  done  to  increase  the  number  of  errors  for  the  majority  of  participants  who  had 
made  such  a  low  number  of  errors  in  Experiment  1.  Based  on  performance  and  on 
participants’  reactions,  these  scenarios  were  much  more  complicated  than  those  in 
Experiment  1.  Participants  experienced  more  errors  in  this  experiment  than  in  the  last.  As 
in  Experiment  1,  performance  was  correlated  with  perceived  workload.  Participants 
indicated  a  higher  level  of  perceived  workload  the  more  poorly  they  had  performed  during 
the  sessions. 

Several  of  the  participants’  comments  indicated  that  they  had  trouble  taking  over  control  of 
scenarios  originally  controlled  by  others.  This  may  have  resulted  because  the  experimental 
conditions  did  not  provide  enough  time  to  allow  for  adequate  preparation  of  strategies  or 
because  it  was  difficult  to  work  with  a  scenario  that  had  been  based  on  someone  else’s 
strategies.  Planning  did  appear  to  be  very  disrupted  in  this  experiment.  Several  participants 
did  not  use  even  simple  control  commands  (data  tags,  holds)  to  help  them  manage  the  traffic 
more  effectively.  As  was  the  case  for  Experiment  1,  it  was  not  possible  to  determine 
individual  planning  strategies,  and  these  may  have  been  critical  in  distinguishing  performance 
levels. 

Experiment  3  was  designed  to  determine  the  effect  of  note  writing  on  performance  and  on 
memory  for  critical  information.  The  results  indicated  that  flight  strip  use  was  related  both 
to  performance  and  to  memory  for  issued  commands.  Additionally,  the  results  indicated  that 
as  performance  and  memory  increased,  perceived  workload  decreased.  Those  who  used 
flight  strips  more,  perform^  better  and  also  felt  that  the  task  was  less  demanding. 

Since  correlations  were  used  to  analyze  these  data,  further  investigations  are  needed  to 
determine  whether  note  writing  produced  better  memory  for  issued  commands,  or  whether 
flight  strips  are,  for  example,  simply  used  more  elaborately  by  participants  who  are  generally 
more  competent,  who  develop  better  strategies,  and  who  then  have  the  time  to  devote  to  such 
additional  tasks.  Flight  strip  use  may  be  a  component  of  effective  strategies  as  previously 
suggested  (e.g.,  Hopkin,  1990).  Given  the  present  results,  the  fact  that  flight  strip 
management  is  so  frequently  cited  as  useful  for  maintaining  awareness  in  the  ATC 
environment,  and  that  this  technique  has  considerable  controller  acceptance,  additional 
investigations  into  the  usefulness  of  flight  strips  appear  worthwhile. 

Over  the  course  of  the  three  experiments,  two  patterns  emerged  that  are  worth  noting.  For 
one,  factors  with  which  participants  entered  the  experiments  as  identified  by  the  preliminary 
questionnaire  (reported  stress  and  aviation  course  experience)  were  associate  only  with 
performance  in  the  first  experiment.  None  were  correlated  with  performance  on  the  second 
or  third  experiments.  This  suggests  that  the  additional  practice  obtained  after  participating  in 
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subsequent  experiments  affected  performance  to  a  greater  extent  than  did  the  factors  that 
participants  initially  brought  with  them. 

Also  noteworthy  was  that  performance  was  correlated  with  perceived  workload  in  all  of  the 
experiments.  Participants  who  performed  better,  also  reported  that  their  level  of  perceived 
workload  was  lower  than  those  who  performed  more  poorly.  This  was  found  even  in 
Experiment  3  in  which  participants  manually  recorded  all  of  their  commands  in  addition  to 
controlling  aircraft.  Despite  the  fact  that  note  writing  added  to  their  task,  it  was  nevertheless 
associated  with  better  performance  as  well  as  with  reports  of  lower  perceived  workload. 
Those  participants  who  wrote  more  on  strips  had  better  performance  scores  and  felt  the 
session  to  be  less  demanding. 

Finally,  this  work  demonstrates  that  a  PC-based  simulator  can  provide  a  suitable  environment 
for  testing  issues  in  ATC.  Realistic  scenarios  can  be  developed  and  can  be  structured  to 
accommodate  a  wide  range  of  performance  capabilities.  Fewer  aircraft  and  fewer  potential 
conflicts  can  be  programmed  into  scenarios  to  test  novices  or  poorer  performers,  while  more 
difficult  scenarios  can  be  programmed  to  test  those  who  are  more  experienced.  Such 
realism  and  flexibility  makes  this  a  potentially  suitable  testing  device  for  actual  air  traffic 
controllers.  Additionally,  the  fact  that  scenarios  can  be  programmed  to  accommodate 
different  performance  levels  is  especially  useful  for  testing  participants  individually  or  for 
testing  them  over  time.  Programmed  scenarios  can  be  continuously  updated  to  accommodate 
the  current  capabilities  of  each  participant.  Matching  the  scenarios  to  each  participant’s 
capability  level  would  reduce  the  amount  of  variability  observed  between  participants  and 
would  allow  the  effects  of  experimental  conditions  to  be  determined  more  easily. 
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APPENDIX  A 


DOCUMENTS  AND  TRAINING  MATERIALS 


January  6,  1992 


Joseph  Blasenstein 
Coordinator,  Aviation  Program 
Department  of  Commerce 
Mercer  County  Community  College 
Trenton,  NJ  08690*-0182 

Dear  Joe, 

Thank  you  for  agreeing  to  allow  the  MCCC  Aviation  Program  to 
participate  in  the  air  traffic  controller  memory  studies  that  PERI 
will  be  conducting  for  the  FAA.  PERI  will  be  following  the 
requirements  of  the  American  Psychological  Association  concerning 
the  use  of  human  subjects  in  experimental  research  which  are 
outlined  as  follows: 

1)  Subjects  must  be  informed  as  to  the  general  purpose  of  the 
experiments  and  the  task  requirements,  including  the  length  of  time 
they  are  expected  to  participate. 

2)  Subjects  must  be  volunteers  who  are  free  to  withdraw  from  the 
experiments  at  any  time  without  penalty. 

3)  Subjects  who  are  students  must  be  informed  that  participation 
in  the  experiments  and  subsequent  performance  measures  do  not  in 
any  way  relate  to  or  affect  course  grades,  academic  standing,  or 
enrollment.  Participation  must  not  conflict  with  academic 
obligations. 

4)  Subjects  must  be  informed  as  to  whether  participation  in  the 
experiments  involves  any  forseeable  physical  or  psychological  risk 
or  danger.  (NONE) 

5)  Subject  privacy  must  be  maintained.  No  individual  names  or 
identities  may  be  revealed  in  any  reports.  (Of  course,  as  a 
group,  the  Aviation  Program  and  MCCC  will  be  fully  acknowledged  for 
their  contribution  to  the  project.) 

6)  Subjects  must  read  and  sign  a  consent  form  to  indicate  that 
they  understand  all  the  Information  above  before  they  may 
participate. 

To  indicate  your  agreement  to  let  PERI  conduct  the  proposed 
research  experiments  under  the  terms  described  above,  I  ask  that 
you  and  Dean  Sanders  sign  and  return  one  copy  of  this  letter  to  me 
at  the  address  below.  Thank  you  in  advance  for  your  cooperation 
and  I  will  speak  to  you  again  soon  regarding  further  details. 

Sincerely, 

Carolina  Zln^le  ^ 

Printed  Names: 

P _ S  ^  _ 

alf  -f.K. ^ h  3U  a rt’ f _ 


Princeton  Economic  Research.  Inc. 

322  Wall  Straat  •  Princeton  NJ  08540  ^ 

Telephone:  609-924-6891  •  FAX;  609>e83-400e  VA 
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January  21,  1992 


To;  Mercer  County  Community  College  Aviation  Student?. 

PERI '  FAA  Air  Traffic  Controller  Memory  Enhancement 

PERI  (Princeton  Economic  Research  Incorporated)  will  be  conducting  preliminary  research  studies  for 
the  FAA  at  Mercer  County  Community  College  during  the  Spring  ’92  semester.  These  studies  will  test 
the  effectiveness  of  certain  memory  strategies  on  air  traffic  control  performance.  Since  pilots  need  to 
interact  with  and  rely  on  air  traffic  controllers,  any  aid  to  their  performance  benefits  you  too!  Your  input 
in  these  studies  will  be  instrumental  in  helping  us  to  better  understand  what  factors  enhance  air  traffic 
control  performance. 

We  are  selecting  aviation  students  to  participate  in  this  research  because  of  your  interest  in  and  knowledge 
of  the  field.  Direct  knowledge  of  or  experience  with  air  traffic  control  is  not  expected.  In  our  studies 
we  will  be  using  an  air  traffic  control  "game"  that  runs  on  a  PC  and  will  teach  you  all  you  need  to  know. 
We  will  allow  you  some  time  to  practice  with  it  and  then  keep  track  of  your  performance  under  a  few 
different  conditions  to  determine  when  the  job  is  handled  most  effectively.  We  would  also  like  to  get 
feedback  from  you  directly  on  what  you  think  makes  things  easy  or  hard  and  what  kinds  of  mental 
strategies  you  are  using  to  help  you. 

Your  participation  in  these  experiments  is  strictly  voluntary.  You  may  decide  to  withdraw  at  any  time 
without  penalty.  The  work  is  not  related  in  any  way  to  your  academic  performance  or  to  course  grades. 
We  will  be  holding  these  experiments  during  free  time  -  Tuesdays  and  Thursdays  from  11:00  to  12:30 
in  room  BS-317  beginning  the  first  week  of  February  and  continuing  until  April  2  if  necessary.  You  will 
not  be  asked  to  come  to  every  session  we  hold!  If  you  decide  to  participate,  the  total  number  of  sessions 
you  would  attend  would  be  about  10.  That  would  include  everything  *  practice  and  experiment 
participation.  In  addition,  for  many  of  the  sessions  it  would  not  be  necessary  for  you  to  attend  for  the 
entire  1  1/2  hours. 

If  you  decide  you  might  be  interested  in  learning  more  please  leave  your  name  with  Joseph  Blasenstein 
and  come  to  our  introductory  meeting  which  will  be  held  on  Tuesday,  January  28th  at  11:00  in  room 
BS-306.  You  will  learn  more  about  the  details  of  the  experiments  and  will  be  able  to  ask  questions  before 
you  decide  whether  to  sign  up.  Thanks  in  advance  for  your  interest  and  assistance. 

Sincerely, 

C,  - 

Carolina  Zingale,  Ph.b. 


Princeton  Economic  Research,  Inc. 

322  Wall  Street  •  Princeton  NJ  08540  ^ 

Telephone:  609-924-8e91  •  FAX:  609-683-4006 
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PERI  -  FAA  Project:  Air  Traffic  Controller  Memory  Enhancement 


CQNSEHT  FORM; 

My  signature  below  Indicates  my  agreement  to  participate  in 
experiments  investigating  the  role  of  various  memory  strategies  on 
the  performance  of  air  traffic  controllers.  These  experiments  will 
be  conducted  by  personnel  from  Princeton  Economic  Research, 
Incorporated  (PERI)  for  the  Federal  Aviation  Administration  (FAA) . 
I  understand  that  in  these  experiments,  participants  will  be 
working  with  the  TRACON  II  Air  Traffic  Control  Simulator  for  the 
pc.  They  will  be  allowed  practice  to  become  efficient  working  with 
this  software  package,  so  that  they  can  effectively  learn  how  to 
obtain  information  from  the  display  and  issue  commands  from  the 
keyboard.  I  understand  that  practice  with  the  simulator  is 
necessary  and  is  expected  to  take  4  to  5  hours.  These  sessions  are 
to  be  conducted  during  free  time  -  Tuesdays  and  Thursdays  from 
11:00  to  12:30  in  room  BS-317. 

Three  experiments  will  be  conducted.  Each  is  expected  to  take 
about  2  hours.  These  will  again  be  conducted  during  the  Tuesday 
and  Thursday  11  -  12:30  time  slot  in  BS-317.  During  testing, 
performance  under  various  conditions  will  be  monitored  and  scored. 
Participants  will  also  be  asked  to  indicate  their  own  assessment  of 
their  performance  following  each  test  session.  In  addition,  they 
will  be  asked  to  indicate  the  kinds  of  techniques  they  used  to  help 
their  performance.  Statistical  analyses  will  then  be  conducted  on 
these  data  in  order  to  determine  whether  performance  differs 
between  the  test  conditions  and  what  factors  contribute  to  those 
differences. 

I  understand  that  my  participation  in  these  studies  is 
strictly  voluntary  and  I  may  withdraw  at  any  time  without  penalty. 
My  participation  and  performance  do  not  in  any  way  relate  to  or 
affect  my  course  grades  or  academic  standing.  I  also  understand 
that  my  right  to  privacy  will  be  protected.  No  individual  names  or 
identities  are  ever  released  in  any  reports.  The  contribution  of 
the  MCCC  Aviation  Program  will,  however,  be  fully  acknowledged.  I 
understand  that  there  are  no  forseeable  physical  or  psychological 
risks  associated  with  participation  in  these  studies. 


Signed: 

Print: 

Date: 
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PRELIMINARY  QUESTIONNAIRE 

The  purpose  of  this  questionnaire  is  to  find  out  something  about 
your  background  and  current  feelings  about  this  project  in  order  to 
better  understand  your  performance  during  the  course  of  the  study. 
All  information  is  collected  under  your  code  number  and  no  attempt 
will  be  made  to  link  your  name  to  the  answers  you  provide. 

1.  Participant  code:  _ 

2 .  Age ;  _ 

3.  Semesters  in  MCCC  aviation  program:  _ 

4.  Number  of  basic  ground  courses  completed:  _ 

5.  Number  of  flight  hours  completed:  _ 

6.  Have  you  had  any  aviation  experience  prior  to  your  enrollment 
at  MCCC?  yes  no 

If  yes,  what  and  for  how  long? 


7.  Using  the  scale 

provided. 

rate  your 

level 

of  computer 

experience. 

1 

2 

3 

4  5 

6  7 

8 

9 

10 

none 

extensive 

8 .  Rate 

your 

level  of 

video  game 

experience. 

1 

2 

3 

4  5 

6  7 

8 

9 

10 

none 

extensive 

9 .  Rate 

your 

current 

vision. 

1 

2 

3 

4  5 

6  7 

8 

9 

10 

poor 

excellent 

10.  Do  you  have  a  color  vision  deficiency?  yes  no 
If  so,  what  is  it? 
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11.  I  freely  volunteered  to  participate 
1  2  3  4  5  6 

strongly 
disagree 


in  this  project 
7  8  9 


10 

Strongly 

agree 


12.  I  am  currently  in  good  health. 

12345678 

Strongly 

disagree 


9  10 

strongly 
agree 


If  not,  are  you  currently  taking  any  medication  that  Interferes 
with  your  vision  or  thinking  abilities?  yes  no 


13.  During  the  last  several  months, 
relatively  high  level  of  stress. 

1  2  3  4  5  6 

strongly 
disagree 


I  have  been  experiencing  a 


7 


8  9  10 

Strongly 

agree 


14.  I  am  not  very  motivated  to  participate 
1  2  3  4  5  6  7 

strongly 
disagree 


in  this  study. 

8  9  10 


strongly 

agree 
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EXPERIMENT  PARTICIPATION 


II  II II  II II  M 

MU  1  1  III! 

1 

+ 

+ 

+ 

2 

+ 

+ 

+ 

3 

+ 

+ 

+ 

4 

+ 

+ 

— 

5 

+ 

+ 

+ 

6 

+ 

+ 

+ 

7 

+ 

+ 

+ 

8 

+ 

+ 

— 

9 

+ 

+ 

+ 

10 

+ 

+ 

+ 

11 

+ 

+ 

+ 

12 

+ 

+ 

+ 

13 

+ 

— 

+ 

14 

— 

— 

+ 

15 

+ 

+ 

participated 

— 

did  not  participate 
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TRAINING  MANUAL  FOR  TESTING 

COMMONLY  USED  TERMS,  ACRONYMNS  AND  ABBREVIATIONS  ON  FLIGHT  PROGRESS 
STRIPS . 


What  is  a  "Flight  Progress  Strip?" 


A  flight  progress  strip  is  a  vnritten  record  of  an  aircrafts 
performance  as  it  either  takes-off ,  lands  or  overflies  the  airspace 
controlled  by  the  controller  i YOU ^ .  A  typical  flight  plan  follows: 


■ 

00 

1o1 

I 

O' 


loo 


/HDDS 


Dou/a/B  L.ifx 


WHAT  DOES  THIS  MEAN????? 


SK234  - Scandinavian  Airlines  flight  niunber  234 

707 - Type  of  aircraft.  A  Boeing  707 

250k - Airspeed  250  knots 

0001— —Time  of  arrival  into  problem 

100' - -Altitude  10,000  feet. (The  last  two  zeros  are 

always  dropped  in  a  flight  plan. 


MIDDS -  An  intersection  of  two  routes  used  by  pilots 

when  they  navigate. 

V165— — —  A  highway  in  the  sky.  Victor  165 
DOWNE - —  Another  intersection. 

LAX  Twr -  The  airport  of  intended  landing,  Los  Angeles. 


PENDING  Flight  progress  strips  or  flight  plans  that  are 

inactive.  The  "pending"  file  will  appear  about  five 
minutes  prior  to  the  flights  needing  control  action. 
(PENDING  FILES  ARE  BLUE  ) 


ACTIVE  Flight  progress  strips  or  flight  plans  that  are  active 
and  under  your  control. 

(ACTIVE  FILES  ARE  GREEN) 


SELECTEIL.  Flight  progress  strip  that  you  are  currently  issuing 
instructions  to. 

(SELECTED  FILES  ARE  BLACK) 


THE  (Tower)  Flight  plans  Indicating  "TWR"  are  aircraft  destined  to 
land  at  the  airport  or  "twr"  specified.  Those  airports 

are:  LAX - -Los  Angeles 

VNY - Van  Nuys 

LGB— — Long  Beach 
TOA — - — Torrance 
SMO—— Santa  Monica 
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CTR  (Center)  Flight  plans  indicating  "CTR"  are  aircraft  overflying 
the  Los  Angeles  airspace  enroute  to  another  "ctr" 
sector.  All  identifiers  for  "ctr"  control  have  five 
(5)  letter  characters,  e.g.  MIDDS,  HASSA 


T/0.  (Take-off)  Flight  plans  indicating  "T/0"  are  aircraft  taking 

off  from  one  of  the  airports  indicated  under 
"twr” .  These  aircraft  are  going  from  a  "twr"  to 
"ctr”  environment. 

*  (Asterisk)  Flight  plans  having  an  in  front  of  the 

routing  indicates  that  control  action  on  that 
flight  plan  is  needed. 


CONTROL  ACTIONS  NECESSARY  TO  SAFELY  MANAGE  TRAFFIC  IN  THE  LOS 
ANGELES  SECTOR 


1) . . OVERFLIGHTS  Route  of  flights  that  start  with  a  five  letter 

identifiers  such  as  MIDDS  and  end  with  "Ctr".  You  must  take  a  hand- 
off  from  the  center  controller  when  the  aircraft  flashes  or  blinks 
at  you,  monitor  the  aircrafts  flight  through  your  airspace, 
protecting  it  from  other  flights  at  the  same  altitude  and  initiate 
a  hand-off  to  another  center  controller  when  the  flight  is  five  fsi 
miles  or  less  from  the  last  five  letter  identifier  or  sector 
boundary . 


RULES ; 


1.  Take  hand-offs  as  soon  as  possible 

2.  Keep  other  aircraft  at  the  same  altitude  at  least  3 
miles  away  from  each  other. 

3.  If  aircraft  are  less  than  3  miles  from  each  other,  you 
must  have  at  least  1,000  ft.  separation  between 
aircraft. 

4.  Make  final  hand-off  when  the  flight  is  five  miles  or 
less  from  the  sector  boundary. 


2 ) .  DEPARTURES  Route  Of  flights  that  start  with  T/0  and  end  with 
"Ctr".  These  aircraft  are  on  the  ground  (pending)  at  the  various 
airports  waiting  for  you  to  release  them  (activate  the  flight 
plan).  Departure  aircraft  need  to  be  released,  separated  from  other 
active  traffic,  monitored  to  a  point  five  miles  from  the  sector 
boundary  (last  five  letter  identifier)  and  then  handed-off  to  the 
next  center  controller. 
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RULES ; 


1.  Release  aircraft  as  soon  as  possible. 

2.  Make  sure  you  have  at  least  3  miles  vertical  or 
1,000ft.  horizontal  separation  from  all  other  traffic 
in  your  sector. 

3 .  Insure  that  aircraft  are  at  the  flight  planned  altitude 
prior  to  making  your  hand-off. 

4.  Insure  that  aircraft  are  going  to  the  proper  exit  fix. 

5.  Make  final  hand-off  when  the  flight  is  five  miles  or 
less  from  the  sector  boundary. 


3 ) .  ARRIVALS  Route  of  flights  that  end  with  "Tvnr".  These  aircraft 
are  the  most  difficult  to  control  since  you  must  take  a  hand-off 
from  the  center  controller,  radar  vector  or  maneuver  the  aircraft 
to  the  appropriate  final  approach  fix,  descend  the  aircraft  to  the 
proper  altitude,  turn  the  aircraft  on  the  final  approach  course  and 
clear  the  aircraft  to  contact  the  tower. 


RULES; 


1.  Take  hand-offs  as  soon  as  possible 

2.  Make  sure  you  have  at  least  3  miles  vertical  or  1000  ft. 
horizontal  separation  from. all  other  aircraft  in  your 
sector. 

3.  Descend  aircraft  to  the  proper  final  descent  altitude. 

4.  Radar  vector  or  maneuver  the  aircraft  towards  the  final 
approach  course. 

5.  Turn  aircraft  on  to  the  final  approach  course  outside 
the  final  approach  fix  (F.A.F.)  on  a  heading  no  greater 
than  thirty  degrees  from  the  final  approach  heading  as 
indicated  on  the  airports  chart. 

6.  Make  final  hand-off  to  the  tower  prior  to  the  F.A.F. 


4 \ .  TOWER  EN-ROUTES  Route  Of  flights  that  start  with  T/0  and  end 
with  "Twr". These  aircraft  are  on  the  ground  at  one  airport,  waiting 
to  take  off  and  land  at  another  airport  in  your  sector.  You  must 
release  or  activate  the  flight  as  indicated  under  DEPARTURES,  and 
then  follow  the  instructions  pertaining  to  vectoring  as  listed  in 
ARRIVALS . 

RULES:  1.  Release  the  aircraft  as  soon  as  possible. 

2.  Follow  instructions  under  ARRIVALS  listed  above. 
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COMMONLY  USED  TERMINOLOGY 


Request  Vector - Aircraft  is  requesting  assistance  to  the  airport 

or  to  a  navigational  fix. 

Request  descent -  Aircraft  is  getting  close  to  the  airport  of 

intended  landing  without  having  had  a 
clearance  to  descend  to  the  final  approach 
altitude. 

Request  release -  Tower  controller  is  asking  permission  for  a 

flight  at  his/her  airport  to  fly  into  your 
sector  under  instrument  flight  rules. 

Missed  approach -  Aircraft  that  you  cleared  for  an  approach  at 

one  of  your  airports  cannot  make  a  safe  landing 
due  to  being  either  too  high  ,  too  close  to  the 
airport,  too  far  away  from  the  final  approach 
course,  or  being  at  the  wrong  altitude. 

Not  on  my  scope  yet - Center  controller  reminding  you  that  you  are 

handing  the  aircraft  off  outside  the  5  mile 
parameter  recognized  by  the  game. 


ttQ_W  TO  COMMUNICATE  TO  AIRCRAFT  AND  OTHER  CONTROLLERS 


In  order  to  communicate  to  aircraft  or  other  controllers,  you  must 
take  three  specific  steps. 

STEP  ONE -  SELECT  AN  AIRCRAFT. 

When  an  aircraft,  control  tower  or  center  controller  wants  you  to 
assume  control  of  an  aircraft  (or  select  the  aircraft),  you'll  hear 
the  request,  see  the  aircraft  blinking  at  you  and  see  the  request 
written  on  your  PC  at  the  bottom  of  your  screen.  (Pink  area)  You 
can  select  the  aircraft  by  either  of  the  following  means. 

a)  Scroll  ARROW  UP  or  ARROW  DOWN  to  HIGHLIGHT  THE  AIRCRAFT,  THEN 
PRESS  THE  ENTER  KEY. 

b)  TYPE  THE  AIRCRAFT  IDENTIFICATION  When  the  App/Dep.  prompt 
appears  in  the  pink  area,  THEN  PRESS  THE  ENTER  KEY. 


You  now  have  selected  the  aircraft  that  you  wish  to  control. 
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STEP  TWO - L-r—  ISSUE  A  COMMAND  INSTRUCTION. 


Issue  to  your  selected  aircraft  the  appropriate  connand 
instructions  as  listed  in  the  AIRCRAFT  SELECTED  COLUMN  below. 


(KEYBOARD  ENTRIES) 


AIRCRAFT  SELECTED  NO  AIRCRAFT  SELECTED 


ARROW  UP - Climb  and  maintain  _  Scroll  up 

ARROW  DOWN - Descend  and  maintain _  Scroll  dotm 

ARROW  LEFT— - Turn  left _  *************** 

ARROW  RIGHT- — -Turn  right _  *************** 

BACKSPACE - Disregard  previous  command  Cancel  last  entry 

SPEED  — ———Change  speed  to  _  ***************** 

DIRECT  TO  ——Cleared  direct  to  _  ***************** 

SAY  HEADING— —Say  heading  and  airspeed  ***************** 

RESUME  NORMAL— Resume  speed  and  o%m  navigation  ***************** 
HAND-OFF— —Hand-off  to  CTR  or  TWR  ***************** 

HOLD  AT  ---—Hold  at  (designated  fix)  ***************** 

ENTER - - - Release  traffic  on  ground  Select  aircraft 

Take  hand-off  from  center  (most  important) 

SEMI-COLON  (;)-To  issue  multiple  commands  ***************** 

e . g .  Command  ( ; )  Command 

PLUS  (+)— — Zoom  in  ***************** 

MINUS  ( — ) — — — — — — Zoom  out  ***************** 

SLANT  (/)— — -Move  aircraft  leader  to——  ***************** 


STEP  THREE - Define  the  specific  parameters  such  as  altitude  or 

heading  using  the  numbers  functions  on  the  left  keyboard. 

ALTITUDE - The  last  two  digits  of  the  altitude  are  always  omitted. 

e.g.  19  means  1900ft.,  120  means  12,000ft. 

HEADING - To  turn  to  a  specific  number  of  degrees  use  two  digits. 

e.g.  20  means  alter  heading  twenty  degrees. 

To  turn  to  a  specific  heading  use  three  digits. 

e.g.  020  means  heading  Zero  Two  Zero  degrees. 


HQB-JTQ  TAKE  A  RADAR  HAND-QFF  AND  RELEASE  A  DEPARTURE  AIRCRAFT. 

This  process  only  requires  two  steps. 

Step  One - Select  the  aircraft  and  hit  the  ENTER  KEY 

Step  two - Depress  the  ENTER  KEY  for  the  second  time. 
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OTHER  INFORMATION  KEYBOARD  ENTRIES. 

These  entries  are  provided  for  your  information  and  are  useful  to 
obtain  additional  information. 


ALT+F -  To  gain  access  to  the  TRACON  menu. 

ARROWS - To  move  left  or  right,  up  or  down  in  the  menu. 

TAB -  To  move  the  cursor  into  a  different  field. 

ESC - To  cancel  last  entry. 

CTRL+F -  To  show  the  aircrafts  flight  plan  route. 

CTRL+T -  To  show  the  Airport  information. 

CTRL+A -  To  show  aircraft  performance  characteristics. 


TO  ACTIVATE  ANY  INFORMATION  REQUESTS  YOU  MUST  FOLLOW-UP  BY 
DEPRESSING  THE  ENTER  KEY. 


HOW  TO  NAXZNIZB  YOUR  SCGRIMG. 

1)  Take  hand-off's  as  soon  as  possible. 

2)  Keep  all  aircraft  at  the  saae  altitude  at  least  3  miles  or  1,000 
feet  away  from  each  other. 

3)  Turn  aircraft  on  to  the  final  am»^c^  course  outside  of  the 
ai^roach  gate  or  course  indicator. 

4)  Hand-off  aircraft  no  sooner  than  5  miles  from  the  sector 
boundary. 

5)  Don't  turn  or  maneuver  aircraft  unnecessarily. 

6)  Don't  forget  to  make  hand-off's  to  the  next  center  controller. 

7)  Turn  aircraft  onto  the  approach  course  at  the  proper  altitude 
and  at  a  heading  that  does  not  differ  by  more  than  thirty 
degrees  from  the  heading  on  your  cAiart. 

8)  Release  departure  aircraft  as  soon  as  possible. 

9)  Try  to  issue  multiple  commands  using  the  (J)  to  allow  you  to 
control  the  frequency. 

10)  DON'T  AIN  TWO  AIRCRAFT  AT  BACH  OTRBR.  A  COLLISION  IS  AN 
AUTONATIC  BXIT  FRON  TBB  TBST. 


This  testing  material  will  be  fully  explained  to  you  by  your 
instructor  who  was  a  qualified  Air  Traffic  Controller.  Feel  free  to 
ask  any  questions  about  the  information  hand-out.  Air  Traffic 
Control  in  general,  or  memory  strategies  we  hope  to  teach  you. 

Remember,  we  expect  you  to  make  mistakes.  We  want  you  to  have 
"FUN".  Try  to  do  your  best  but  don't  worry  if  you  get  behind  or 
can't  remember  everything.  You  are  not  expected  to  become  Air 
Traffic  controller  after  this  experiment. 

Thanks  again  for  your  volunteer  participation. 

The  staff  at  PERI. 
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control  instruction  commands 
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PRACTICE  SESSION; 

Part  1.  Based  on  the  advance  Information  you  now  have  from  the 
flightplans  of  the  aircraft  in  the  upcoming  scenario,  what  actions 
can  you  preplan  that  will  be  useful  for  you  in  managing  the  air 
traffic  flow? 


Part  2,  Now  write  out  specific  instructions  next  to  each 
aircraft's  flightstrip  to  use  as  reminders  during  the  session. 

For  example,  reminders  to  change  altitude,  or  speed  or  route. 
Write  your  instructions  in  the  appropriate  columns  of  the 
f lightstrips. 


In  a  few  moments  the  practice  session  will  begin  and  you  will 
be  asked  to  manage  the  aircraft,  making  as  few  errors  as  possible. 
To  remind  yourself  of  the  aircraft's  flightplan  during  the  session, 
use  the  FLIGHTPLAN  command  (ALT  D  then  F) . 
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Experiment  l  —  <  PLANNING  >. 

General  instructions  (Practice  session). 

Before  beginning  the  upcoming  scenario,  you  will  be  provided 
with  the  flightstrips  for  all  the  aircraft  that  will  come  under 
your  control  during  the  session.  You  will  then  have  about  10 
minutes  to  use  this  information  to  your  best  advantage: 
to  plan  strategies  for  handling  these  aircraft  in  advance  so  that 
your  errors  can  be  minimized. 

To  give  you  a  feel  for  how  this  can  be  done  effectively,  you 
will  now  have  an  opportunity  to  practice  with  the  following.  The 
flightstrips  presented  on  the  handout  describe  the  aircraft  that 
will  enter  your  airspace  in  a  short  session.  In  the  third  column 
of  the  flightstrip  is  the  time  that  you  can  expect  that  aircraft  to 
look  to  come  under  your  control.  For  example,  N16FG,  N9565G,  and 
NlEQ  will  look  to  make  contact  within  1  minute  of  the  start  of  the 
scenario.  AA123  and  SK190  will  look  to  make  contact  within  2 
minutes . 

Before  you  begin  actually  directing  this  traffic,  think  about 
the  way  things  will  look  at  different  points  in  the  scenario  and 
how  you  would  anticipate  handling  any  potential  problems. 

(a)  N16FG  will  enter  first.  It  is  entering  at  LANGE 
(find  this  location  on  the  map).  It  is  going  to  go  from 
here  to  MIDDS  center  (check  location)  .  Note  that  it  win 
be  travelling  at  146  knots  at  an  altitude  of  8000  feet 
when  it  enters  your  airspace. 

(b)  Now,  look  at  N9565G.  It  will  enter  yu^  airspace 
just  at  about  the  same  time  as  N16FG.  It  is  entering  at 
SAUGS  and  will  be  landing  at  Long  Beach  airport.  It  will 
initially  be  travelling  at  a  speed  of  220  knots  at  an 
altitude  of  9000  feet.  If  you  haven't  done  so  already, 
locate  SAUGS  on  the  map.  You  will  see  that  LANGE  and 
SAUGS  are  very  close  to  one  another. 

(c)  Now  check  the  flightpaths  of  both  aircraft.  The  way 
things  are  set  up,  FG  and  5G  will  be  travelling  along 
almost  parallel  paths,  close  to  one  another.  You  now 
know  that  you  will  need  to  be  continuously  aware  of  the 
2  aircraft's  proximity  to  one  another.  This  lets  you 
plan  in  advance  how  you  will  keep  them  separated  during 
the  scenario.  Keep  in  mind  that  you  must  control  the 
aircraft  as  efficiently  as  possible.  For  example,  having 
them  deviate  a  long  way  from  their  intended  routes  is  not 
the  most  efficient  strategy  and  you  will  see  that  points 
will  be  deducted  from  your  score. 

Now  work  with  some  of  the  other  information.  What  do  the 
flightpaths  look  like?  Are  there  any  potential  conflicts 
that  you  can  forsee? 
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KiODS-  VhS^ 


SAUCrS'MlDOS 


CODE: 


PERFORMANCE  CHECK  LIST 


CRASH 

SEPERATION 

AIRCRAFT 

MISSED 

HANDOFF  AT 

HANDOFF 

CONFLICT 

VECTORED 

OF  SCOPE 

APPROACH 

WRONG 

ALTITUDE 

NOT  MADE 

TO  CENTER 

END  OF  SESSION  SUMMARY  (TOTALS) 

CATEGORY  NUMBER 

SEPARATION  CONFLICTS  _ 

MISSED  APPROACHED  _ 

HANDOFF  ERRORS  _ 

PILOT  REQUESTS  granted/ total  _  /__ 

SCORE: _ 
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CONTROLLER  SIMULATION  QUESTIONNAIRE 


PLEASE  COMPLETE  THE  FOLLOWING  QUESTIONS  AS  SOON  AS  YOU  HAVE 
COMPLETED  THE  SESSION.  VOUR  RESPONSES  SHOULD  FOCUS  ON  ONLY  THE 
WORK  THAT  YOU  HAVE  JUST  COMPLETED  IN  THE  LAST  CONTROL  PERIOD. 

ALL  CONTROLLERS  EXPERIENCE  A  WIDE  VARIETY  OF  ACTIVITY  AND  RESULTANT 
WORKLOAD  DURING  THEIR  CAREERS.  IT  DOES  NOT  DETRACT  FROM  YOUR 
PROFpSIONALISM  IF  FOR  A  GIVEN  PERIOD  YOU  REPORT  VERY  HIGH  OR 
VERYLOW  WORKLOAD.  ON  ALL  THE  QUESTIONS  WHICH  FOLLOW  FEEL  FREE  TO 
USE  THE  ENTIRE  NUMERICAL  SCALE  FOR  EACH  ANSWER.  BE  AS  HONEST  AND 
ASACCURATE  AS  YOU  CAN.  YOUR  NAME  IS  NOT  RECORDED  ON  THIS  OR  ANY 
OTHER  FORM,  AND  NO  ATTEMPT  WILL  BE  MADE  TO  ASSOCIATE  YOUR  RESPONSES 
WITH  YOU  AS  AN  INDIVIDUAL.  DATA  COLLECTED  WILL  BE  FOR  RESEARCH 
PURPOSES  ONLY. 
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Participant  code:  _ 

1.  Choose  the  one  number  below  which  best  describes  how  hard  you 
were  wor)(ing  during  this  period: 

DESCRIPTION  OF  WORKLOAD  RATING 

_ CATEGORY _ fCIRCLE  ONE^ 


VERY  LOW  WORKLOAD  -  All  tasks  were  1 

accomplished  quickly  and  easily  2 

_ 2_ 

MODERATE  WORKLOAD  -  The  chances  for  4 

errors  or  omissions  were  low  5 

_ &. 

RELATIVELY  HIGH  WORKLOAD  -  The  chances  7 

for  some  errors  or  omissions  were  8 

_ relatively  high _ 2. 

VERY  HIGH  WORKLOAD  -  It  was  barely  10 

possible  to  accomplish  all  tasks  11 

properly  12 


2.  Rate  your  performance  controlling  traffic  during  the  past 
session,  circle  the  number  which  best  describes  how  well  you  think 
you  did. 

123456789  10 
average  excellent 


3.  What  fraction  of  the  time  were 
were  controlling? 

1  2  3  4  5 

seldom 
had  much 
to  do 


you  busy  during  the  period  you 


6  7  8  9  10 

fully 

occupied  at 
all  times 


4.  How  much  did  you  have  to  think  during  this  period? 


minimal 
thinking 
and  concen'- 
tration 
required 


5  6  7  8  9  10 

a  great  deal 
of  thinking 
and  concen** 
tration 
required 
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5.  Rate  the 
stressful . 

degree 

to  which 

you  found 

this 

control 

period 

1  2 
low 

3 

4  5 

6  7 

8 

9 

10 

high 

6.  I  am  feeling  tired. 

123456789 


10 


strongly 

disagree 


strongly 

agree 


]7.  Briefly  describe  your  strategy  for  working  traffic  during  this 
control  period. 


8.  If  you  have  a  choice  of  separating  aircraft  vertically  or 
horizontally,  which  do  you  prefer  to  do  and  why? 


9.  Is  thelTe  anything  else  that  happened  this  past  session  which  you 
feel  might  help  us  understand  the  results?  Any  comments  you  have 
at  this  point  would  be  very  welcome. 
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CONTROL  INSTRUCTIONS  ISSUED 


DIRECT  HAND  CLIMBS/  SPEED  CLEAR 
CUIAR  OFFS  DESCENTS  RSTRNS  APC. 


HOLD/ 

OTHER 
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